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Foreword 


A few years ago, during a trip to Milan for a Red Hat event, I ran into a passionate 
colleague at the Red Hat office. We spoke at length about how customers in Italy 
adopt containers to speed up application development on OpenShift. While his name 
slipped my mind at the time, his enthusiasm about the subject didn't, especially since 
he was also hospitable enough to take me to an espresso bar near the office to show 
me what real coffee tastes like. A while later, I was introduced to a developer advocate 
in a meeting who would speak at a conference about CI/CD using products like 
OpenShift Pipelines and OpenShift GitOps that my teams delivered at the time. At 
that moment, I instantly recognized Natale. Many who attended that talk thought 
it was insightful, given his firsthand grasp of challenges that customers experience 
when delivering applications and his hands-on approach to technology. 


Application delivery is a complex process involving many systems and teams with 
numerous handoffs between these parties, often synonymous with delays and back- 
and-forth talks at each point. Automation has long been a key enabler for improving 
this process and has become particularly popular within the DevOps movement. 
Continuous integration, infrastructure as code, and numerous other practices became 
common in many organizations as they navigated their journey toward adopting 
DevOps. 


More recently, and coinciding with the increased adoption of Kubernetes, GitOps as 
a blueprint for implementing a subset of DevOps practices has become an area I fre- 
quently get asked about. While neither the term nor the practices GitOps advocates 
are new, it does combine. It presents the existing knowledge in a workflow that is 
simple, easy to understand, and can be implemented in a standard way across many 
teams. 


xi 


Although the path to adopting the GitOps workflow is simple and concrete, many 
technical choices need to be made to fit within each organization's security, compli- 
ance, operational, and other requirements. Therefore, I am particularly thrilled about 
the existence of this book and the practical guides it provides to assist these teams in 
making choices that are right for their applications, teams, and organizations. 


—Siamak Sadeghianfar 
Product Management, Red Hat 
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Preface 


We wrote this book for builders. Whether you are a developer, DevOps engineer, 
site reliability engineer (SRE), or platform engineer dealing with Kubernetes, you are 
building some good stuff. We would like to share our experience from what we have 
learned in the field and in the community about the latest Kubernetes automation 
insights for pipelines and CI/CD workloads. The book contains a comprehensive list 
of the most popular available software and tools in the Kubernetes and cloud native 
ecosystem for this purpose. We aim to provide a list of practical recipes that might 
help your daily job or are worth exploring further. We are not sticking to a particular 
technology or project for implementing Kubernetes automation. However, we are 
opinionated on some of our choices to deliver a concise GitOps pathway. 


The book is organized in sequential chapters, from the basics to advanced topics in 
the Kubernetes ecosystem, following the GitOps principles. We hope you'll find these 
recipes valuable and inspiring for your projects! 


e Chapter 1 is an introduction to GitOps principles and why they are continuously 
becoming more common and essential for any new IT project. 


e Chapter 2 covers the installation requirements to run these recipes in a Kuber- 
netes cluster. Concepts and tools like Git, Container Registry, Container Run- 
time, and Kubernetes are necessary for this journey. 


e Chapter 3 walks you through a complete overview of containers and why they 
are essential for application development and deployment today. Kubernetes is a 
container-orchestration platform; however, it doesn't build containers out of the 
box. Therefore, we'll provide a list of practical recipes for making container apps 
with the most popular tools available in the cloud native community. 


e Chapter 4 gives you an overview of Kustomize, a popular tool for managing 
Kubernetes resources. Kustomize is interoperable, and you find it often used 
within CI/CD pipelines. 


xiii 


Chapter 5 explores Helm, a trendy tool to package applications in Kubernetes. 
Helm is also a templating system that you can use to deploy apps in CI/CD 
workloads. 


Chapter 6 walks you through cloud native CI/CD systems for Kubernetes. It gives 
a comprehensive list of recipes for the continuous integration part with Tekton, 
the Kubernetes-native CI/CD system. Additionally, it also covers other tools such 
as Drone and GitHub Actions. 


Chapter 7 kicks off the pure GitOps part of the book as it sticks to the Continu- 
ous Deployment phase with Argo CD, a popular GitOps tool for Kubernetes. 


Chapter 8 goes into the advanced topics for GitOps with Argo CD, such as 
secrets management, progressive application delivery, and multicluster deploy- 
ments. This concludes the most common use cases and architectures you will 
likely work with today and tomorrow following the GitOps approach. 


Conventions Used in This Book 


The following typographical conventions are used in this book: 


Ttalic 
Indicates new terms, URLs, email addresses, filenames, and file extensions. 


Constant width 
Used for program listings, as well as within paragraphs to refer to program 
elements such as variable or function names, databases, data types, environment 
variables, statements, and keywords. 


Constant width bold 
Shows commands or other text that should be typed literally by the user. 


Constant width italic 
Shows text that should be replaced with user-supplied values or by values deter- 
mined by context. 


This element signifies a tip or suggestion. 


This element signifies a general note. 
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This element indicates a warning or caution. 


Using Code Examples 


Supplemental material (code examples, exercises, etc.) is available for download at 
https://github.com/gitops-cookbook. 


If you have a technical question or a problem using the code examples, please send 
email to bookquestions@oreilly.com. 


This book is here to help you get your job done. In general, if example code is 
offered with this book, you may use it in your programs and documentation. You 
do not need to contact us for permission unless you're reproducing a significant 
portion of the code. For example, writing a program that uses several chunks of code 
from this book does not require permission. Selling or distributing examples from 
O'Reilly books does require permission. Answering a question by citing this book 
and quoting example code does not require permission. Incorporating a significant 
amount of example code from this book into your product’s documentation does 
require permission. 


We appreciate, but generally do not require, attribution. An attribution usually 
includes the title, author, publisher, and ISBN. For example: “GitOps Cookbook by 
Natale Vinto and Alex Soto Bueno (O'Reilly). Copyright 2023 Natale Vinto and Alex 
Soto Bueno, 978-1-492-09747-1? 


If you feel your use of code examples falls outside fair use or the permission given 
above, feel free to contact us at permissions@oreilly.com. 


O'Reilly Online Learning 


N » For more than 40 years, O’Reilly Media has provided technol- 
O REILLY ogy and business training, knowledge, and insight to help 


companies succeed. 


Our unique network of experts and innovators share their knowledge and expertise 
through books, articles, and our online learning platform. O’Reilly’s online learning 
platform gives you on-demand access to live training courses, in-depth learning 
paths, interactive coding environments, and a vast collection of text and video from 
O'Reilly and 200+ other publishers. For more information, visit http://oreilly.com. 
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How to Contact Us 


Please address comments and questions concerning this book to the publisher: 


OReilly Media, Inc. 

1005 Gravenstein Highway North 

Sebastopol, CA 95472 

800-998-9938 (in the United States or Canada) 
707-829-0515 (international or local) 
707-829-0104 (fax) 


We have a web page for this book, where we list errata, examples, and any additional 
information. You can access this page at hittps://oreil.ly/gitops-cookbook. 


Email bookquestions@oreilly.com to comment or ask technical questions about this 
book. 


For news and information about our books and courses, visit https://oreilly.com. 
Find us on LinkedIn: https://linkedin.com/company/oreilly-media. 
Follow us on Twitter: https://twitter.com/oreillymedia. 


Watch us on YouTube: hitps://youtube.com/oreillymedia. 
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CHAPTER 1 
Introduction 


With the advent of practices such as infrastructure as code (IaC), software develop- 
ment has pushed the boundaries of platforms where you can run applications. This 
becomes more frequent with programmable, API-driven platforms such as public 
clouds and open source infrastructure solutions. While some years ago developers 
were only focusing on application source code, today they also have the opportunity 
to code the infrastructure where their application will run. This gives control and 
enables automation, which significantly reduces lead time. 


A good example is with Kubernetes, a popular open source container workload 
orchestration platform and the de facto standard for running production applica- 
tions, either on public or private clouds. The openness and extensibility of the 
platform enables automation, which reduces risks of delivery and increases service 
quality. Furthermore, this powerful paradigm is extended by another increasingly 
popular approach called GitOps. 


1.1 What Is GitOps? 


GitOps is a methodology and practice that uses Git repositories as a single source 
of truth to deliver infrastructure as code. It takes the pillars and approaches from 
DevOps culture and provides a framework to start realizing the results. The relation- 
ship between DevOps and GitOps is close, as GitOps has become the popular choice 
to implement and enhance DevOps, platform engineering, and SRE. 


GitOps is an agnostic approach, and a GitOps framework can be built with tools such 
as Git, Kubernetes, and CI/CD solutions. The three main pillars of GitOps are: 


e Git is the single source of truth 


e Treat everything as code 


e Operations are performed through Git workflows 


There is an active community around GitOps, and the GitOps Working Group 
defines a set of GitOps Principles (currently in version 1.0.0) available at 
OpenGitOps: 


Declarative 
A system managed by GitOps must have its desired state expressed declaratively. 


Versioned and immutable 
The desired state is stored in a way that enforces immutability and versioning 
and retains a complete version history. 


Pulled automatically 
Software agents automatically pull the desired state declarations from the source. 


Continuously reconciled 
Software agents continuously observe the actual system state and attempt to 
apply the desired state. 


1.2 Why GitOps? 


Using the common Git-based workflows that developers are familiar with, GitOps 
expands upon existing processes from application development to deployment, app 
lifecycle management, and infrastructure configuration. 


Every change throughout the application lifecycle is traced in the Git repository and 
is auditable. This approach is beneficial for both developers and operations teams 
as it enhances the ability to trace and reproduce issues quickly, improving overall 
security. One key point is to reduce the risk of unwanted changes (drift) and correct 
them before they go into production. 


Here is a summary of the benefits of the GitOps adoption in four key aspects: 


Standard workflow 
Use familiar tools and Git workflows from application development teams 


Enhanced security 
Review changes beforehand, detect configuration drifts, and take action 


Visibility and audit 
Capture and trace any change to clusters through Git history 
Multicluster consistency 


Reliably and consistently configure multiple environments and multiple Kuber- 
netes clusters and deployment 
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1.3 Kubernetes CI/CD 


Continuous integration (CI) and continuous delivery (CD) are methods used to fre- 
quently deliver apps by introducing automation into the stages of app development. 
CI/CD pipelines are one of the most common use cases for GitOps. 


In a typical CI/CD pipeline, submitted code checks the CI process while the CD 
process checks and applies requirements for things like security, infrastructure as 
code, or any other boundaries set for the application framework. All code changes are 
tracked, making updates easy while also providing version control should a rollback 
be needed. CD is the GitOps domain and it works together with the CI part to deploy 
apps in multiple environments, as you can see in Figure 1-1. 


Deplo Deplo 


Continuous Integration 


Continuous Delivery 


Figure 1-1. Continuous integration and continuous delivery 


With Kubernetes, it’s easy to implement an in-cluster CI/CD pipeline. You can have 
CI software create the container image representing your application and store it in a 
container image registry. Afterward, a Git workflow such as a pull request can change 
the Kubernetes manifests illustrating the deployment of your apps and start a CD 
sync loop, as shown in Figure 1-2. 


CI : 
Source Git Repository o— a: eB Image Registry 
(m) i 


Pull Request 


v 


Y cD 
Config Git Repository — © > f: Kubernetes 


Figure 1-2. Application deployment model 


This cookbook will show practical recipes for implementing this model on Kuber- 
netes acting as a CI/CD and GitOps platform. 
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1.4 App Deployment with GitOps on Kubernetes 


As GitOps is an agnostic, platform-independent approach, the application deploy- 
ment model on Kubernetes can be either in-cluster or multicluster. An external 
GitOps tool can use Kubernetes just as a target platform for deploying apps. At the 
same time, in-cluster approaches run a GitOps engine inside Kubernetes to deploy 
apps and sync manifests in one or more Kubernetes clusters. 


The GitOps engine takes care of the CD part of the CI/CD pipeline and accomplishes 
a GitOps loop, which is composed of four main actions as shown in Figure 1-3: 


Deploy 
Deploy the manifests from Git. 


Monitor 
Monitor either the Git repo or the cluster state. 


Detect drift 
Detect any change from what is described in Git and what is present in the 
cluster. 


Take action 
Perform an action that reflects what is on Git (rollback or three-way diff). Git is 
the source of truth, and any change is performed via a Git workflow. 


Monitor 


Figure 1-3. GitOps loop 


In Kubernetes, application deployment using the GitOps approach makes use of at 
least two Git repositories: one for the app source code, and one for the Kubernetes 
manifests describing the apps deployment (Deployment, Service, etc.). 


Figure 1-4 illustrates the structure of a GitOps project on Kubernetes. 
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Push soi * © Ge Pipelines Pusti Image 
Registry 
© © ful 
Pull-request j Pull i ; Sync 
ull-reques Git GitOps engine 
app manifests 


Figure 1-4. Kubernetes GitOps loop 


The following list outlines the items in the workflow: 


. App source code repository 
. CI pipeline creating a container image 
. Container image registry 


. Kubernetes manifests repository 


nN A OU N Fe 


. GitOps engine syncing manifests to one or more clusters and detecting drifts 


1.5 DevOps and Agility 


GitOps is a developer-centric approach to continuous delivery and infrastructure 
operations, and a developer workflow through Git for automating processes. As 
DevOps is complementary to Agile software development, GitOps is complementary 
to DevOps for infrastructure automation and application lifecycle management. As 
you can see in Figure 1-5, it’s a developer workflow for automating operations. 


One of the most critical aspects of the Agile methodology is to reduce the lead 
time, which is described more abstractly as the time elapsed between identifying a 
requirement and its fulfillment. 
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Deployment 
(clusters, apps) 


Monitoring 
Tracing 


Loggin: 
(observability) 


Continuous GIT GitOps for 


Integration (and image repo) Kubernetes 


Management 
(operations) 


‘Immutability firewall’ 


Figure 1-5. GitOps development cycle 


Reducing this time is fundamental and requires a cultural change in IT organizations. 
Seeing applications live provides developers with a feedback loop to redesign and 
improve their code and make their projects thrive. Similarly to DevOps, GitOps also 
requires a cultural adoption in business processes. Every operation, such as applica- 
tion deployment or infrastructure change, is only possible through Git workflows. 
And sometimes, this means a cultural shift. 


The “Teaching Elephants to Dance (and Fly!)” speech from Burr Sutter gives a clear 
idea of the context. The elephant is where your organization is today. There are 
phases of change between traditional and modern environments powered by GitOps 
tools. Some organizations have the luxury of starting from scratch, but for many 
businesses, the challenge is teaching their lumbering elephant to dance like a graceful 
ballerina. 
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CHAPTER 2 
Requirements 


This book is about GitOps and Kubernetes, and as such, you'll need a container 
registry to publish the containers built throughout the book (see Recipe 2.1). 


Also, a Git service is required to implement GitOps methodologies; you'll learn how 
to register to public Git services like GitHub or GitLab (see Recipe 2.2). 


Finally, it would be best to have a Kubernetes cluster to run the book examples. 
Although we'll show you how to install Minikube as a Kubernetes cluster (see Recipe 
2.3), and the book is tested with Minikube, any Kubernetes installation should work 
as well. 


Lets prepare your laptop to execute the recipes provided in this book. 


2.1 Registering for a Container Registry 


Problem 


You want to create an account for a container registry service so you can store 
generated containers. 


Solution 


You may need to publish some containers into a public container registry as you work 
through this book. Use Docker Hub (docker . io) to publish containers. 


If you already have an account with docker.io, you can skip the following steps. 
Otherwise, keep reading to learn how to sign up for an account. 


Discussion 
Visit DockerHub to sign up for an account. The page should be similar to Figure 2-1. 


Build and Ship any 
. . Get Started Today for Free 
Application Anywhere Mient hare cine Sn 


Docker ID 


Docker Hub is the world's easiest way to create, manage, 
and deliver your teams' container applications. 


Email 


Password © 


C Send me occasional product updates and announcements. 


[o [agree to the Subscription Service Agreement, Privacy Policy 
and Data Processing Terms. 


C tmpot rot e 


Figure 2-1. DockerHub registration page 
When the page is loaded, fill in the form by setting a Docker ID, Email, and Pass- 
word, and click the Sign Up button. 


When you are registered and your account confirmed, you'll be ready to publish 
containers under the previous step’s Docker ID. 


See Also 


Another popular container registry service is quay.io. It can be used on the cloud (like 
docker.io) or installed on-premises. 


Visit the website to get more information about Quay. The page should be similar to 
Figure 2-2. 
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@RED HAT Quay.io EXPLORE TUTORIAL PRICING search SIGN IN 


|Quay.io now supports Red Hat Single Sign-On Services exclusively. If you haven't done so already, you need to link your Quay.io login to a redhat.com account in order to be able to login to the web interface by going to th 


endpoint. CLI tokens and robot accounts are not impacted. Read more about this change in the FAQ. 


Quay [ builds, analyzes, distributes ] your container images 


TRY FOR FREE ON PREMISES TRY FOR FREE ON CLOUD 


@)podman S$ oocker Qrkt 


Figure 2-2. Quay registration page 
2.2 Registering for a Git Repository 


Problem 
You want to create an account for a Git service so you can store source code in a 
repository. 
Solution 


You may need to publish some source code into a public Git service in this book. Use 
GitHub as a Git service to create and fork Git repositories. 


If you already have an account with GitHub, you can skip the following steps, 
otherwise keep reading to learn how to sign up for an account. 


Discussion 


Visit the GitHub web page to sign up for an account. The page should be similar to 
Figure 2-3. 
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©) whveittubz. Team Enterprise Explore Marketplace Pricing — Search GitHub Signin [ Sign up 


Where the world 
builds software 


Millions of developers and companies build, ship, and | 
maintain their software on GitHub—the largest and 
most advanced development platform in the world. 


73+ million 4+ mill: 200+ million 84% 


Developer 


Figure 2-3. GitHub welcome page to register 


When the page is loaded, click the Sign up for GitHub button (see Figure 2-3) and 
follow the instructions. The Sign in page should be similar to Figure 2-4. 


ount? Sign in > 


Welcome to GitHub! 
Let's begin the adventure 


Enter your email 
>i 


Figure 2-4. Sign In GitHub page 


When you are registered and your account confirmed, you'll be ready to start creating 
or forking Git repositories into your GitHub account. 
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Also, you'll need to fork the book si 
Fork button shown in Figure 2-! 


e code repository into your account. Click the 


<> Code © Issues [9 Pull requests 


P main ~ P 2branches © 0 tags 


sÊ. lordofthejars Adds PR example 


a apps/bubbles-frontend 
W cho 
@ chos 


D -gitignore 


© Actions 


A gitops-cookbook / gitops-cookbook-sc ‘Public 


EB Projects 


Adds Bubbles 
Adds PR example 
Adds PR example 


Adds Bubbles 


Help people interested in this repository understand your project by adding a README. 


QO Search or jump to... (7) Pullrequests Issues Marketplace Explore 


& Edit Pins ~ 


© Security Lx Insights 


Goto file 


Add file ~ (code) 


3e63da1 27 daysago ©20 commits 


3 months ago 
27 days ago 
27 days ago 


3 months ago 


© Unwatch D ~ 


wy starf 


About 


No description, website, or topics 
provided. 


Yr Istar 
© 1watching 
Y 1 fork 


Releases 


No releases published 
Create a new release 


Figure 2-5. Fork button 


Then select your account in the Owner section, if not selected yet, and click the 
button “Create fork” button as shown in Figure 2-6. 


Create a new fork 


A fork is a copy of a repository. Forking a repository allows you to freely experiment with changes without 
affecting the original project. View existing forks. 


Repository name * 
gitops-cookbook-sc 


By default, forks are named the same as their parent repository. You can customize the name to distinguish it 
further. 


Description (optional) 


@ Copy the main branch only 
Contribute back to gitops-cookbook/gitops-cookbook-sc by adding your own branch. Learn more. 


@ You are creating a fork in your personal account. 


D 


Figure 2-6. Create fork button 
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To follow along with the example in the following chapters, you can clone this book’s 
repositories locally. When not mentioned explicitly, we will refer to the examples 
available in the chapters repo: 


git clone https://github.com/gitops-cookbook/chapters 


See Also 


Another popular Git service is GitLab. It can be used on the cloud or installed 
on-premises. 


Visit GitLab for more information. 


2.3 Creating a Local Kubernetes Cluster 


Problem 


You want to spin up a Kubernetes cluster locally. 


Solution 


In this book, you may need a Kubernetes cluster to run most recipes. Use Minikube 
to spin up a Kubernetes cluster in your local machine. 


Discussion 


Minikube uses container/virtualization technology like Docker, Podman, Hyperkit, 
Hyper-V, KVM, or VirtualBox to boot up a Linux machine with a Kubernetes cluster 
installed inside. 


For simplicity and to use an installation that will work in most of the platforms, we 
are going to use VirtualBox as a virtualization system. 


To install VirtualBox (if you haven't done it yet), visit the home page and click the 
Download link as shown in Figure 2-7. 


For those using macOS, the following instructions have been tested 
on a Mac AMD64 with macOS Monterey and VirtualBox 6.1. At 
the time of writing this book, there were some incompatibilities 
when using the ARM version or macOS Ventura. 
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VirtualBox is a powerful x86 and AMD64/Intel64 virtualization product for enterprise as well as home use. Not only is VirtualBox an extremely 
feature rich, high performance product for enterprise customers, it is also the only professional solution that is freely available as Open Source 
nate Software under the terms of the GNU General Public License (GPL) version 2. See "About VirtualBox" for an introduction. 

out 

Presently, VirtualBox runs on Windows, Linux, Macintosh, and Solaris hosts and supports a large number of guest operating systems including but 


Screenshots not limited to Windows (NT 4.0, 2000, XP, Server 2003, Vista, Windows 7, Windows 8, Windows 10), DOS/Windows 3.x, Linux (2.4, 2.6, 3.x and 
Downloads] 4.x), Solaris and OpenSolaris, OS/2, and OpenBSD. 
Documentation VirtualBox is being actively developed with frequent releases and has an ever growing list of features, supported guest operating systems and 
End. d platforms it runs on. VirtualBox is a community effort backed by a dedicated company: everyone is encouraged to contribute while Oracle ensures 
Diigo the product always meets professional quality criteria. 


Technical docs 
Contribute 


citi Download 


VirtualBox 6.1 


Hot picks: 


+ Pre-built virtual machines for developers at Oracle Tech Network 
+ Hyperbox Open-source Virtual Infrastructure Manager & project site 
+ phpVirtualBox AJAX web interface => project site 


Figure 2-7. VirtualBox home page 


Select the package based on the operating system, download it, and install it on your 
computer. After installing VirtualBox (we used the 6.1.x version), the next step is to 
download and spin up a cluster using Minikube. 


Visit the GitHub repo, unfold the Assets section, and download the Minikube file that 
matches your platform specification. For example, in the case of an AMD Mac, you 
should select minikube-darwin-amd64 as shown in Figure 2-8. 


Uncompress the file (if necessary) and copy it with the name minikube in a directory 
accessible by the PATH environment variable such as (/usr/local/bin) in Linux or 
macOS. 


With VirtualBox and Minikube installed, we can spin up a Kubernetes cluster in the 
local machine. Let’s install Kubernetes version 1.23.0 as it was the latest version at the 
time of writing (although any other previous versions can be used as well). 
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v Assets 49 
@ docker-machine-driver-hyperkit 8.35 MB 
@ docker-machine-driver-hyperkit.sha256 65 Bytes 
@ docker-machine-driver-kvm2 11.4 MB 
@ docker-machine-driver-kvm2-1.24.0-0.x86_64.rpm 3.35 MB 
@ docker-machine-driver-kvm2-amd64 11.4 MB 
@ docker-machine-driver-kvm2-amd64.sha256 65 Bytes 
@ docker-machine-driver-kvm2-arm64 11 MB 
© dóckermachine-driver Kim2anmgA:sha266 65 Bytes 
© dockermachihe -diver kim? x86 64 11.4 MB 
@ docker-machine-driver-kvm2.sha256 65 Bytes 
is) docker-machine-driver-kvm2_1.24.0-0_amd64.deb 5.01 MB 
@ docker-machine-driver-kvm2_1.24.0-0_arm64.deb 4.47 MB 
@ minikube-1.24.0-0.aarch64.rpm 25.1MB 
@ minikube-1.24.0-0.armv7hl.rpm 25 MB 
@ minikube-1.24.0-0.ppc64le.rpm 24.5 MB 
@ minikube-1.24.0-0.s390x.rpm 26.6 MB 
@ minikube-1.24.0-0.x86_64.rpm 15 MB 
@ minikube-darwin-amd64 ; 65.7 MB 
@ minikube-darwin-amd64.sha256 65 Bytes 
@ minikube-darwin-amd64.tar.gz 30.1 MB 


Figure 2-8. Minikube release page 


Run the following command in a terminal window to spin up the Kubernetes cluster 
with 8 GB of memory assigned: 


minikube start --kubernetes-version='v1.23.0' / 
--driver='virtualbox' --memory=8196 -p gitops 000 


@ Creates a Kubernetes cluster with version 1.23.0 
© Uses VirtualBox as virtualization tool 


© Creates a profile name (gitops) to the cluster to refer to it later 
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The output lines should be similar to: 


[gitops] Minikube v1.24.0 on Darwin 12.0.1 

Using the virtualbox driver based on user configuration 
Starting control plane node gitops in cluster gitops (1) 
Creating virtualbox VM (CPUs=2, Memory=8196MB, Disk=20000MB) 


> kubeadm.sha256: 64 B / 64 B [-------------------------- ] 100.00% ? p/s Os 
> kubelet.sha256: 64 B / 64 B [-------------------------- ] 100.00% ? p/s Os 
> kubectl.sha256: 64 B / 64 B [-------------------------- ] 100.00% ? p/s Os 
> kubeadm: 43.11 MiB / 43.11 MiB [--------------- ] 100.00% 3.46 MiB p/s 13s 
> kubectl: 44.42 MiB / 44.42 MiB [--------------- ] 100.00% 3.60 MiB p/s 13s 
> kubelet: 118.73 MiB / 118.73 MiB [------------- ] 100.00% 6.32 MiB p/s 19s 


=" Generating certificates and keys ... 

" Booting up control plane ... (2) 

* Configuring RBAC rules ... 

" Using image gcr.io/k8s-minikube/storage-provisioner:v5 


Verifying Kubernetes components... 
Enabled addons: storage-provisioner, default-storageclass 
/usr/local/bin/kubectl is version 1.21.0, which 
may have incompatibilites with Kubernetes 1.23.0. © 
" Want kubectl v1.23.0? Try 'minikube kubectl -- get pods -A' 


Done! kubectl is now configured to use "gitops" cluster and 
"default" namespace by default (4) 


Starts the gitops cluster 
Boots up the Kubernetes cluster control plane 


Detects that we have an old kubectl tool 


© © 8 8 


Cluster is up and running 


To align the Kubernetes cluster and Kubernetes CLI tool version, you can download 
the kubectl 1.23.0 version running from https://dl.k8s.io/release/v1.23.0/bin/darwin/ 
amd64/kubectl. 


You need to change darwin/amd64 to your specific architecture. For 
example, in Windows it might be windows /amd64/kubectl.exe. 


Copy the kubectl CLI tool in a directory accessible by the PATH environment variable 
such as (/usr/local/bin) in Linux or macOS. 
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See Also 
There are other ways to run Kubernetes in a local machine. 
One that is very popular is kind. 


Although the examples in this book should work in any Kubernetes implementation 
as only standard resources are used, weve only tested with Minikube. 
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CHAPTER 3 
Containers 


Containers are a popular and standard format for packaging applications. The format 
is an open standard promoted by the Open Container Initiative (OCI), an open gov- 
ernance structure for the express purpose of creating open industry standards around 
container formats and runtimes. The openness of this format ensures portability 
and interoperability across different operating systems, vendors, platforms, or clouds. 
Kubernetes runs containerized apps, so before going into the GitOps approach to 
managing apps on Kubernetes, we provide a list of recipes useful for understanding 
how to package your application as a container image. 


The first step for creating images is to use a container engine for packaging your 
application by building a layered structure containing a base OS and additional layers 
on top such as runtimes, libraries, and applications. Docker is a widespread open 
source implementation of a container engine and runtime, and it can generate a 
container image by specifying a manifest called a Dockerfile (see Recipe 3.1). 


Since the format is open, it’s possible to create container images with other tools. 
Docker, a popular container engine, requires the installation and the execution of a 
daemon that can handle all the operations with the container engine. Developers can 
use a software development kit (SDK) to interact with the Docker daemon or use 
dockerless solutions such as JiB to create container images (see Recipe 3.2). 


If you dont want to rely on a specific programming language or SDK to build 
container images, you can use another daemonless solution like Buildah (see Recipe 
3.3) or Buildpacks (see Recipe 3.4). Those are other popular open source tools for 
building OCI container images. By avoiding dependencies from the OS, such tools 
make automation more manageable and portable (see Chapter 6). 


7 


Kubernetes doesn't provide a native mechanism for building container images. How- 
ever, its highly extensible architecture allows interoperability with external tools and 
the platform’s extensibility to create container images. Shipwright is an open source 
framework for building container images on Kubernetes, providing an abstraction 
that can use tools such as kaniko, Buildpacks, or Buildah (see Recipe 3.5) to create 
container images. 


At the end of this chapter, you'll learn how to create OCI-compliant container images 
from a Dockerfile, either from a host with Docker installed, or using tools such as 
Buildah and Buildpacks. 


3.1 Building a Container Using Docker 


Problem 


You want to create a container image for your application with Docker. 


Solution 
The first thing you need to do is install Docker. 
Docker is available for Mac, Windows, and Linux. Download the 


installer for your operating system and refer to the documentation 
to start the Docker service. 


Developers can create a container image by defining a Dockerfile. The best definition 
for a Dockerfile comes from the Docker documentation itself: “A Dockerfile is a text 
document that contains all the commands a user could call on the command line to 
assemble an image.” 


Container images present a layered structure, as you can see in Figure 3-1. Each 
container image provides the foundation layer for a container, and any update is just 
an additional layer that can be committed on the foundation. 
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F Image Layer 3 
p Image Layer 2 


Application Layer 


i Java Runtime Layer 
i Image Layer 1 OS Update Layer 


Base RHEL 


Container Image Layers Example Container Image 


Figure 3-1. Container image layers 


You can create a Dockerfile like the one shown here, which will generate a container 
image for Python apps. You can also find this example in this book’s repository. 


© © 8 Ọ 


FROM registry.access.redhat.com/ubi8/python-39 (13 
ENV PORT 8080 @ 

EXPOSE 8080 © 

WORKDIR /usr/src/app (4) 


COPY requirements.txt ./ (5) 
RUN pip install --no-cache-dir -r requirements. txt 6] 


COPY a a 


ENTRYPOINT ["python"] @ 
CMD ["app.py"] © 


FROM: always start from a base image as a foundational layer. In this case we start 
from a Universal Base Image (UBI), publicly available based on RHEL 8 with 
Python 3.9 runtime. 

ENV: set an environment variable for the app. 

EXPOSE: expose a port to the container network, in this case port TCP 8080. 


WORKDIR: set a directory inside the container to work with. 


COPY: copy the assets from the source code files on your workstation to the 
container image layer, in this case, to the WORKDIR. 


RUN: run a command inside the container, using the tools already available within 
the base image. In this case, it runs the pip tool to install dependencies. 


ENTRYPOINT: define the entry point for your app inside the container. It can be a 
binary or a script. In this case, it runs the Python interpreter. 
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© CMD: the command that is used when starting a container. In this case it uses the 
name of the Python app app. py. 


You can now create your container image with the following command: 
docker build -f Dockerfile -t quay.io/gitops-cookbook/pythonapp: Latest 
Change the container image name with the your registry, user, and 


repo. Example: quay.io/youruser/yourrepo: latest. See Chap- 
ter 2 for how to create a new account on registries such as Quay.io. 


Your container image is building now. Docker will fetch existing layers from a public 
container registry (DockerHub, Quay, Red Hat Registry, etc.) and add a new layer 
with the content specified in the Dockerfile. Such layers could also be available locally, 
if already downloaded, in special storage called a container cache or Docker cache. 


STEP 1: FROM registry.access.redhat.com/ubi8/python-39 
Getting image source signatures 
Copying blob adffa6963146 done 
Copying blob 4125bdfaec5e done 
Copying blob 362566a15abb done 
Copying blob 0661f10c38cc done 
Copying blob 26f1167feaf7 done 
Copying config a531ae7675 done 
Writing manifest to image destination 
Storing signatures 
STEP 2: ENV PORT 8080 
--> 6dbf4ac027e 
STEP 3: EXPOSE 8080 
--> £78357fe402 
STEP 4: WORKDIR /usr/src/app 
--> 547bf8ca5c5 
STEP 5: COPY requirements.txt ./ 
--> 456cab38c97 
STEP 6: RUN pip install --no-cache-dir -r requirements. txt 
Collecting Flask 
Downloading Flask-2.0.2-py3-none-any.whl (95 kB) 
E 95 KB 10.6 MB/s 
Collecting itsdangerous>=2.0 
Downloading itsdangerous-2.0.1-py3-none-any.whl (18 kB) 
Collecting Werkzeug>=2.0 
Downloading Werkzeug-2.0.2-py3-none-any.whl (288 kB) 
E 288 KB 1.7 MB/s 
Collecting click>=7.1.2 
Downloading click-8.0.3-py3-none-any.whl (97 kB) 
E 07 KB 31.9 MB/s 
Collecting Jinja2>=3.0 
Downloading Jinja2-3.0.3-py3-none-any.whl (133 kB) 


E) 133 KB 38.8 MB/s 


STEP 7: COPY . 
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--> 3e6b73464eb 

STEP 8: ENTRYPOINT ["python" ] 

--> acabca89260 

STEP 9: CMD ["app.py"] 

STEP 10: COMMIT quay.io/gitops-cookbook/pythonapp: Latest 

--> 52e134d39af 
52e134d39af013a25f3e44d25133478dc20b46626782762f4e46b1fF F6F0243bb 


Your container image is now available in your Docker cache and ready to be used. 
You can verify its presence with this command: 


docker images 


You should get the list of available container images from the cache in output. Those 
could be images you have built or downloaded with the docker pull command: 


REPOSITORY TAG IMAGE ID CREATED 
SIZE 

quay .io/gitops-cookbook/pythonapp latest 52e134d39afO0 6 minutes agot 
907 MB 


Once your image is created, you can consume it locally or push it to a public 
container registry to be consumed elsewhere, like from a CI/CD pipeline. 


You need to first log in to your public registry. In this example, we are using Quay: 
docker login quay.io 

You should get output similar to this: 
Login Succeeded! 

Then you can push your container image to the registry: 
docker push quay.io/gitops-cookbook/pythonapp: Latest 

As confirmed, you should get output similar to this: 


Getting image source signatures 
Copying blob e6e8a2c58ac5 done 
Copying blob 3ba8c926eef9 done 
Copying blob 558b534f4e1b done 
Copying blob 25f82e0f4ef5 done 
Copying blob 7b17276847a2 done 
Copying blob 352ba846236b done 
Copying blob 2de82c390049 done 
Copying blob 26525e00a8d8 done 
Copying config 52¢134d39a done 
Writing manifest to image destination 
Copying config 52e134d39a [-------------------------------------- ] 0.0b / 5.4KiB 
Writing manifest to image destination 
Storing signatures 
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Discussion 


You can create container images in this way with Docker from your workstation or 
any host where the Docker service/daemon is running. 


Additionally, you can use functionalities offered by a public regis- 
try such as Quay.io that can directly create the container image 
from a Dockerfile and store it to the registry. 


The build requires access to all layers, thus an internet connection to the registries 
storing base layers is needed, or at least having them in the container cache. Docker 
has a layered structure where any change to your app is committed on top of the 
existing layers, so there's no need to download all the layers each time since it will add 
only deltas for each new change. 


Container images typically start from a base OS layer such as 
Fedora, CentOS, Ubuntu, Alpine, etc. However, they can also start 
from scratch, an empty layer for super-minimal images contain- 
ing only the app’s binary. See the scratch documentation for more 
info. 


If you want to run your previously created container image, you can do so with this 
command: 


docker run -p 8080:8080 -ti quay.io/gitops-cookbook/pythonapp: Latest 


docker run has many options to start your container. The most common are: 

-P 
Binds the port of the container with the port of the host running such container. 
Attaches a TTY to the container. 


Goes into an interactive mode. 


Goes in the background, printing a hash that you can use to interact asynchro- 
nously with the running container. 
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The preceding command will start your app in the Docker network and bind it to 
port 8080 of your workstation: 


* 


Serving Flask app 'app' (lazy loading) 

Environment: production 

WARNING: This is a development server. Do not use it in a production deployment. 
Use a production WSGI server instead. 

Debug mode: on 

Running on all addresses. 

WARNING: This is a development server. Do not use it in a production deployment. 
Running on http://10.0.2.100:8080/ (Press CTRL+C to quit) 

Restarting with stat 

Debugger is active! 

Debugger PIN: 103-809-567 


* 


* 


* 


+ + + + 


From a new terminal, try accessing your running container: 
curl http://localhost:8080 
You should get output like this: 


Hello, World! 


See Also 


e Best practices for writing Dockerfiles 


e Manage Docker images 


3.2 Building a Container Using Dockerless Jib 


Problem 


You are a software developer, and you want to create a container image without 
installing Docker or any additional software on your workstation. 


Solution 


As discussed in Recipe 3.1, you need to install the Docker engine to create container 
images. Docker requires permissions to install a service running as a daemon, thus 
a privileged process in your operating system. Today, dockerless solutions are also 
available for developers; a popular one is Jib. 


Jib is an open source framework for Java made by Google to build OCI-compliant 
container images, without the need for Docker or any container runtime. Jib comes 
as a library that Java developers can import in their Maven or Gradle projects. This 
means you can create a container image for your app without writing or maintaining 
any Dockerfiles, delegating this complexity to Jib. 
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We see the benefits from this approach as the following:' 


Pure Java 
No Docker or Dockerfile knowledge is required. Simply add Jib as a plug-in, and 
it will generate the container image for you. 


Speed 
The application is divided into multiple layers, splitting dependencies from 
classes. There’s no need to rebuild the container image like for Dockerfiles; Jib 
takes care of modifying the layers that changed. 


Reproducibility 
Unnecessary updates are not triggered because the same contents generate the 
same image. 


The easiest way to kickstart a container image build with Jib on existing Maven is by 
adding the plug-in via the command line: 
mvn compile com.google.cloud. tools: jib-maven-plugin:3.2.0:build -Dimage=<MY IMAGE> 
Alternatively, you can do so by adding Jib as a plug-in into your pom.xml: 
<project> 
<build> 
<pLugins> 
<plugin> 
<groupId>com.google.cloud.tools</groupId> 
<artifactId>jib-maven-plugin</artifactId> 
<version>3.2.0</version> 
<configuration> 
<to> 
<image>myimage</image> 
</to> 


</configuration> 
</plugin> 


</plugins> 
</build> 
</project> 
In this way, you can also manage other settings such as authentication or parameters 
for the build. 


Let’s now add Jib to an existing Java application, a Hello World application in Spring 
Boot that you can find in the book’s repository. 


1 For a presentation about Jib, see Appu Goundan and Qingyang Chen's presentation from Velocity San Jose 
2018. 
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Run the following command to create a container image without using Docker, and 
push it directly to a container registry. In this example, we use Quay.io, and we will 
store the container image at quay.io/gitops-cookbook/jib-example:latest, so you will 
need to provide your credentials for the registry: 


mvn compile com.google.cloud.tools: jib-maven-plugin:3.2.0:build \ 
-Dimage=quay.io/gitops-cookbook/jib-example:latest \ 
-Djib.to.auth.username=<USERNAME> \ 
-Djib.to.auth.password=<PASSWORD> 


The authentication here is handled with command-line options, but Jib can manage 


existing authentication with Docker CLI or read credentials from your settings.xml 
file. 


The build takes a few moments, and the result is a Java-specific container image, 
based on the adoptOpenJDK base image, built locally and pushed directly to a regis- 
try. In this case, to Quay.io: 


[INFO] 

[INFO] -------------------------- < com.redhat:hello >-------------------------- 
[INFO] Building hello 0.0.1-SNAPSHOT 

[INFO] -------------------------------- [jar Peywtes aieceeeates ranea aE 


INFO] Containerizing application to quay.io/gitops-cookbook/jib-example... 
INFO] Using credentials from <to><auth> for quay.io/gitops-cookbook/jib-example 
INFO] The base image requires auth. Trying again for eclipse-temurin:11-jre... 
INFO] Using base image with digest: b 
sha256:83d92ee225e443580cc3685ef9574582761cf975abc53850c2bc44ec47d7d9430] 

INFO] 

INFO] Container entrypoint set to [java, -cp, @/app/jib-classpath-file,> 
com.redhat.hello.HelloApplication]FO] 


INFO] Built and pushed image as quay.io/gitops-cookbook/jib-example 
INFO] Executing tasks: 
INFO] [+s] 100,0% complete 


INFO] Total time: 41.366 s 
INFO] Finished at: 2022-01-25T19:04:09+01:00 


If you have Docker and run the command docker images, you 
wont see this image in your local cache! 
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Discussion 


Your container image is not present in your local cache, as you don't need any 
container runtime to build images with Jib. You wont see it with the docker images 
command, but you can pull it from the public container registry afterward, and it will 
store it in your cache. 


This approach is suitable for development velocity and automation, where the CI 
system doesn't need to have Docker installed on the nodes where it runs. Jib can 
create the container image without any Dockerfiles. Additionally, it can push the 
image to a container registry. 


If you also want to store it locally from the beginning, Jib can connect to Docker 
hosts and do it for you. 


You can pull your container image from the registry to try it: 


docker run -p 8080:8080 -ti quay.io/gitops-cookbook/jib-example 


Trying to pull quay.io/gitops-cookbook/jib-example: latest... 
Getting image source signatures 
Copying blob ea362f368469 done 
Copying blob d5cc550bb6a0 done 
Copying blob bcc17963ea24 done 
Copying blob 9b46d5d971fa done 
Copying blob 51f4f7c353f0 done 
Copying blob 43b2cdfa19bb done 
Copying blob fd142634d578 done 
Copying blob 78c393914c97 done 
Copying config 346462b8d3 done 
Writing manifest to image destination 
Storing signatures 


IN aa aana NA 
CECA PEVAS EAA 
WW s e LET A 
dek A Aces Ieee 
=========| _|==============| ___/=/_/_/_/ 
:: Spring Boot :: (v2.6.3) 
2022-01-25 18:36:24.762 INFO 1 --- [ main] com.redhat.hello.HelloApplications 


: Starting HelloApplication using Java 11.0.13 on a719cf76f440 with PID 15 
(/app/classes started by root in /) 


2022-01-25 18:36:24.765 INFO 1 --- [ main] com.redhat.hello.HelloApplications 
: No active profile set, falling back to default profiles: default 

2022-01-25 18:36:25.700 INFO 1 --- [ main] 0.s.b.w.embedded. tomcat. TomcatWeb- 
Servers 

: Tomcat initialized with port(s): 8080 (http) 
2022-01-25 18:36:25.713 INFO 1 --- [ main] o.apache.catalina.core.StandardSer - 
viceb 

: Starting service [Tomcat] 
2022-01-25 18:36:25.713 INFO 1 --- [ main] org.apache.catalina.core.StandardEn- 
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gine 
: Starting Servlet engine: [Apache Tomcat/9.0.56] 


2022-01-25 18:36:25.781 INFO 1 --- [ main] 0.a.c.c.C.[Tomcat].[localhost].[/]& 
: Initializing Spring embedded WebApplicationContext 

2022-01-25 18:36:25.781 INFO 1 --- [ main] w.s.c.ServletWebServerApplicationCon- 
textb 

: Root WebApplicationContext: initialization completed in 947 ms 
2022-01-25 18:36:26.087 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWeb- 
Servers 

: Tomcat started on port(s): 8080 (http) with context path '' 
2022-01-25 18:36:26.096 INFO 1 --- [ main] com.redhat.hello.HelloApplications 


: Started HelloApplication in 1.778 seconds (JVM running for 2.177) 
Get the hello endpoint: 


curl localhost:8080/hello 


{"id":1,"content": "Hello, World!"} 


See Also 


e Using Jib with Quarkus projects 


3.3 Building a Container Using Buildah 


Problem 


Sometimes installing or managing Docker is not possible. Dockerless solutions for 
creating container images are useful in use cases such as local development or CI/CD 
systems. 


Solution 


The OCI specification is an open standard, and this favors multiple open source 
implementations for the container engine and the container image building mecha- 
nism. Two growing popular examples today are Podman and Buildah. 


While Docker uses a single monolithic application for creating, 
running, and shipping container images, the codebase for con- 
tainer management functionalities here has been split between 
different projects like Podman, Buildah, and Skopeo. Podman sup- 
port is already available on Mac and Windows, however Buildah 
is currently only available on Linux or Linux subsystems such as 
WSL2 for Windows. See the documentation to install it on your 
workstation. 


Those are two complementary open source projects and command-line tools that 
work on OCI containers and images; however, they differ in their specialization. 
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While Podman specializes in commands and functions that help you to maintain and 
modify container images, such as pulling, tagging, and pushing, Buildah specializes 
in building container images. Decoupling functions in different processes is done by 
design, as the authors wanted to move from the single privileged process Docker 
model to a lightweight, rootless, daemonless, and decoupled set of tools to improve 
agility and security. 


Following the same approach, you find Skopeo, a tool used to 
move container images; and CRI-O, a container engine complaint 
with the Kubernetes container runtime interface for running appli- 
cations. 


Buildah supports the Dockerfile format, but its goal is to provide a lower-level 
interface to build container images without requiring a Dockerfile. Buildah is a 
daemonless solution that can create images inside a container without mounting the 
Docker socket. This functionality improves security and portability since it’s easy to 
add Buildah builds on the fly to a CI/CD pipeline where the Linux or Kubernetes 
nodes do not require a Docker installation. 


As we discussed, you can create a container image with or without a Dockerfile. Lets 
now create a simple HTTPD container image without a Dockerfile. 


You can start from any base image such as CentOS: 
buildah from centos 
You should get output similar to this: 


Resolved short name "centos" to a recorded short-name aliast 
(origin: /etc/containers/registries.conf.d/shortnames.conf) 

Getting image source signatures 

Copying blob 926a85fb4806 done 

Copying config 2f3766df23 done 

Writing manifest to image destination 

Storing signatures 

centos-working-container 


Similarly to Docker and docker images, you can run the com- 
mand buildah containers to get the list of available images from 
the container cache. If you also have installed Podman, this is 
similar to podman images. 


In this case, the container image ID is centos-working-container, and you can refer 
to it for creating the other layers. 


Now let’s install the httpd package inside a new layer: 
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buildah run centos-working-container yum install httpd -y 


You should get output similar to this: 


CentOS Linux 8 - AppStream 9.0 MB/s | 8.4 MB 00:00 
CentOS Linux 8 - BaseOS 436 kB/s | 4.6 MB 00:10 
CentOS Linux 8 - Extras 23 kB/s | 10 kB 00:00 


Dependencies resolved. 


Package Arch Version Repository Size 
Installing: 

httpd x86_64 2.4.37-43.module_el8.5.0+1022+b541f3b1 
Installing dependencies: 

apr x86_64 1.6.3-12.e18 

apr-util x86_64 1.6.1-6.e18 

brotli x86_64 1.0.6-3.e18 

centos-logos-httpd noarch 85.8-2.e18 

httpd- filesystem noarch 2.4.37-43.module_el8.5.0+1022+b541f3b1 
httpd-tools x86_64 2.4.37-43.module_el8.5.0+1022+b541f3b1 
mailcap noarch 2.1.48-3.e18 

mod_http2 x86_64 1.15.7-3.module_el8.4.0+778+c970deab 
Installing weak dependencies: 

apr-util-bdb x86_64 1.6.1-6.e18 

apr-util-openssl x86_64 1.6.1-6.e18 


Enabling module streams: 

Complete! 
Now let’s copy a welcome HTML page inside the container running HTTPD. You can 
find the source code in this book's repo: 


<html> 
<head> 
<title>GitOps CookBook example</title> 
</head> 
<body> 
<hi>Hello, World!</hi> 
</body> 
</html> 


buildah copy centos-working-container index.html /var/www/html/index.html 


For each new layer added, you should get output with the new container image hash, 
similar to the following: 


78c6e1dcd6f819581b54094fd38a3fd8f170a2cb768101e533c964e04aacab2e 


buildah config --entrypoint "/usr/sbin/httpd -DFOREGROUND" centos-working-container 


buildah commit centos-working-container quay.io/gitops-cookbook/gitops-website 
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You should get output similar to this: 


Getting image source signatures 

Copying blob 618ce6bf40a6 skipped: already exists 

Copying blob eb8c13ba832f done 

Copying config b825e91208 done 

Writing manifest to image destination 

Storing signatures 
b825e91208c33371e209cc327abe4f53ee501d5679c127cd71c4d10cd03e5370 


Your container image is now in the container cache, ready to run or push to another 
registry. 


As mentioned before, Buildah can also create container images from a Dockerfile. 
Let’s make the same container image from the Dockerfile listed here: 


FROM centos: latest 

RUN yum -y install httpd 

COPY index.html /var/www/html/index.html 
EXPOSE 80 

CMD ["/usr/sbin/httpd", "-DFOREGROUND" ] 


buildah bud -f Dockerfile -t quay.io/gitops-cookbook/gitops-website 


STEP 1: FROM centos: latest 

Resolved short name "centos" to a recorded short-name alias’ 
(origin: /etc/containers/registries.conf.d/shortnames.conf) 

Getting image source signatures 

Copying blob 926a85fb4806 done 

Copying config 2f3766df23 done 

Writing manifest to image destination 

Storing signatures 

STEP 2: RUN yum -y install httpd 


CentOS Linux 8 - AppStream 9.6 MB/s | 8.4 MB 00:00 
CentOS Linux 8 - BaseOS 7.5 MB/s | 4.6 MB 00:00 
CentOS Linux 8 - Extras 63 kB/s | 10 kB 00:00 


Dependencies resolved. 


Complete! 

STEP 3: COPY index.html /var/www/html/index.html 

STEP 4: EXPOSE 80 

STEP 5: CMD ["/usr/sbin/httpd", "-DFOREGROUND" ] 

STEP 6: COMMIT quay.io/gitops-cookbook/gitops-website 
Getting image source signatures 

Copying blob 618ce6bf40a6 skipped: already exists 
Copying blob 1be523a47735 done 

Copying config 3128caf147 done 

Writing manifest to image destination 

Storing signatures 

--> 3128caf1475 
3128caf147547e43b84c13c241585d23a32601f2c2db80b966185b03cb6a8025 


If you have also installed Podman, you can run it this way: 


podman run -p 8080:80 -ti quay.io/gitops-cookbook/gitops-website 
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Then you can test it by opening the browser on http://localhost:8080. 


Discussion 


With Buildah, you have the opportunity to create container images from scratch 
or starting from a Dockerfile. You don't need to install Docker, and everything 
is designed around security: rootless mechanism, daemonless utilities, and more 
refined control of creating image layers. 


Buildah can also build images from scratch, thus it creates an empty layer similar 
to the FROM scratch Dockerfile statement. This aspect is useful for creating very 
lightweight images containing only the packages needed to run your application, as 
you can see in Figure 3-2. 


Application Layer 


Java Runtime Layer 


OS Update Layer Java runtime and 


dependencies, and 


Base RHEL Application 
ml 

From base, From scratch, 

multilayer single layer 


Figure 3-2. Buildah image shrink 


A good example use case for a scratch build is considering the development images 
versus staging or production images. During development, container images may 
require a compiler and other tools. However, in production, you may only need the 
runtime or your packages. 


See Also 


e Running Buildah inside a container 
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3.4 Building a Container with Buildpacks 


Problem 


Creating container image by using Dockerfiles can be challenging at scale. You want 
a tool complementing Docker that can inspect your application source code to create 
container images without writing a Dockerfile. 


Solution 


Cloud Native Buildpacks is an open source project that provides a set of executables 
to inspect your app source code and to create a plan to build and run your applica- 
tion. 


Buildpacks can create OCI-compliant container images without a Dockerfile, starting 
from the app source code, as you can see in Figure 3-3. 


“>> Dockerfiles 


Source OCI image 


Figure 3-3. Buildpacks builds 


This mechanism consists of two phases: 


Detection 
Buildpacks tooling will navigate your source code to discover which program- 
ming language or framework is used (e.g., POM, NPM files, Python require- 
ments, etc.) and assign a suitable buildpack for the build. 


Building 
Once a buildpack is found, the source is compiled and Buildpacks creates a 
container image with the appropriate entry point and startup scripts. 


To use Buildpacks, you have to download the pack CLI for your operating system 
(Mac, Windows, Linux), and also have Docker installed. 


On macOS, pack is available through Homebrew as follows: 


brew install buildpacks/tap/pack 


32 | Chapter 3: Containers 


Now lets start creating our container image with Buildpacks from a sample Node.js 


app. You can find 


cd chapters/ch 


the app source code in this book’s repository: 


03/nodejs-app 


The app directory structure contains a package.json file, a manifest listing Node.js 
packages required for this build, which helps Buildpacks understand which buildpack 


to use. 


You can verify it with this command: 


pack builder s 


uggest 


You should get output similar to this: 


Suggested buil 
Google 


Heroku 


Heroku 


Paketo 


Paketo 


Paketo 


ders: 

; gcr.io/buildpacks/builder:v1b 

Ubuntu 18 base image with buildpacks for .NET, Go, Java, Node.js,bt 
and Python 

: heroku/buildpacks:184 

Base builder for Heroku-18 stack, based on ubuntu:18.04 base 
image 


: heroku/buildpacks: 204 

Base builder for Heroku-20 stack, based on ubuntu:20.04 baseb 
image 

Buildpacks: paketobuildpacks/builder : base 


Ubuntu bionic base image with buildpacks for Java, .NET Core,& 
Node.js, Go, Python, Ruby, NGINX and Procfile 

Buildpacks: paketobuildpacks/builder: full 

Ubuntu bionic base image with buildpacks for Java, .NET Core,& 
Node.js, Go, Python, PHP, Ruby, Apache HTTPD, NGINX and Procfile 
Buildpacks: paketobuildpacks/builder: tiny 

Tiny base image (bionic build image, distroless-like run image) 
with buildpacks for Java, Java Native Image and Go 


Now you can decide to pick one of the suggested buildpacks. Let’s try the paketo 


buildpacks/buil 


pack build nod 


kn 


der : base, which also contains the Node.js runtime: 


ejs-app --builder paketobuildpacks/builder:base 


Run pack builder inspect paketobuildpacks/builder: base to 


ow the exact content of libraries and frameworks available in this 


buildpack. 


The building process should start accordingly, and after a while, it should finish, and 


you should get ou 


base: Pulling 
bf99a8b93828: 


Digest: sha256 


tput similar to this: 


from paketobuildpacks/builder 
Pulling fs layer 


:7034e52388c11c5f 7ee7ae8f2d7d794ba427cc2802F687dd9650d96a70ac0772 
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Status: Downloaded newer image for paketobuildpacks/builder:base 
base-cnb: Pulling from paketobuildpacks/run 

bf99a8b93828: Already exists 

9d58a4841c3f: Pull complete 

77a4f59032ac: Pull complete 

24e58505e5e0: Pull complete 

Digest: sha256:59aa1da9db6d979e21721e306b9ce99a7c4e3d1663c4c20F74f9b3876cce5192 
Status: Downloaded newer image for paketobuildpacks/run:base-cnb 
===> ANALYZING 

Previous image with name "nodejs-app" not found 

===> DETECTING 

5 of 10 buildpacks participating 
paketo-buildpacks/ca-certificates 3.0.1 

paketo-buildpacks/node- engine 0.11.2 
paketo-buildpacks/npm-install 0.6.2 
paketo-buildpacks/node-module-bom 0.2.0 
paketo-buildpacks/npm-start 0.6.1 

===> RESTORING 

===> BUILDING 


Paketo NPM Start Buildpack 0.6.1 
Assigning launch processes 
web: node server.js 


===> EXPORTING 
Adding layer 'paketo-buildpacks/ca-certificates:helper' 
Adding layer 'paketo-buildpacks/node-engine:node' 
Adding layer 'paketo-buildpacks/npm-install:modules' 
Adding layer 'launch.sbom' 
Adding 1/1 app layer(s) 
Adding layer 'lLauncher' 
Adding layer 'config' 
Adding layer 'process-types' 
Adding label 'io.buildpacks.lifecycle.metadata' 
Adding label 'io.buildpacks.build.metadata' 
Adding label 'io.buildpacks.project.metadata' 
Setting default process type 'web' 
Saving nodejs-app... 
*** Images (82b805699d6b) : 
nodejs-app 
Adding cache layer 'paketo-buildpacks/node-engine:node' 
Adding cache layer 'paketo-buildpacks/npm-install:modules' 
Adding cache layer 'paketo-buildpacks/node-module-bom:cyclonedx-node-module' 
Successfully built image nodejs-app 


Now let’s run it with Docker: 
docker run --rm -p 3000:3000 nodejs-app 
You should get output similar to this: 


Server running at http://0.0.0.0:3000/ 
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View the running application: 
curl http://localhost:3000/ 
You should get output similar to this: 


Hello Buildpacks! 


Discussion 


Cloud Native Buildpacks is an incubating project in the Cloud Native Computing 
Foundation (CNCF), and it supports both Docker and Kubernetes. On Kubernetes, it 
can be used with Tekton, a Kubernetes-native CI/CD system that can run Buildpacks 
as a Tekton Task to create container images. It recently adopted the Boson Project 
to provide a functions-as-a-service (FaaS) experience on Kubernetes with Knative, by 
enabling the build of functions via buildpacks. 


See Also 


e Using Buildpacks with Tekton Pipelines 
e FaaS Knative Boson project’s buildpacks 


3.5 Building a Container Using Shipwright and kaniko 
in Kubernetes 


Problem 


You need to create a container image, and you want to do it with Kubernetes. 


Solution 


Kubernetes is well known as a container orchestration platform to deploy and 
manage apps. However, it doesn’t include support for building container images 
out-of-the-box. Indeed, according to Kubernetes documentation: “(Kubernetes) Does 
not deploy source code and does not build your application. Continuous Integration, 
Delivery, and Deployment (CI/CD) workflows are determined by organization cul- 
tures and preferences as well as technical requirements.” 


As mentioned, one standard option is to rely on CI/CD systems for this purpose, like 
Tekton (see Chapter 6). Another option is to use a framework to manage builds with 
many underlying tools, such as the one we discussed in the previous recipes. One 
example is Shipwright. 
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Shipwright is an extensible framework for building container images on Kubernetes. 
It supports popular tools such as Buildah, Cloud Native Buildpacks, and kaniko. It 
uses Kubernetes-style APIs, and it runs workloads using Tekton. 


The benefit for developers is a simplified approach for building container images, 
by defining a minimal YAML file that does not require any previous knowledge of 
containers or container engines. This approach makes this solution agnostic and 
highly integrated with the Kubernetes API ecosystem. 


The first thing to do is to install Shipwright to your Kubernetes cluster, say kind or 
Minikube (see Chapter 2), following the documentation or from OperatorHub.io. 


Using Operators and Operator Lifecycle Manager (OLM) gives 
consistency for installing/uninstalling software on Kubernetes, 
along with dependency management and lifecycle control. For 
instance, the Tekton Operator dependency is automatically 
resolved and installed if you install Shipwright via the Operator. 
Check the OLM documentation for details with this approach. 


Let’s follow the standard procedure from the documentation. First you need to install 
the Tekton dependency. At the time of writing this book, it is version 0.30.0: 


kubectl apply -f \ 
https: //storage.googleapis.com/tekton-releases/pipeline/previous/v0.30.0/ 
release. yaml 


Then you install Shipwright. At the time of writing this book, it is version 0.7.0: 


kubectl apply -f \ 
https: //github.com/shipwright-io/build/releases/download/v0.7.0/release.yaml 


Finally, you install Shipwright build strategies: 


kubectl apply -f \ 
https: //github.com/shipwright-io/build/releases/download/v0.7.0/sample- 
strategies. yaml 


Once you have installed Shipwright, you can start creating your container image 
build using one of these tools: 


e kaniko 

e Cloud Native Buildpacks 
e BuildKit 

e Buildah 


Let's explore kaniko. 
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kaniko is another dockerless solution to build container images from a Dockerfile 
inside a container or Kubernetes cluster. Shipwright brings additional APIs to Kuber- 
netes to use tools such as kaniko to create container images, acting as an abstract 
layer that can be considered an extensible building system for Kubernetes. 


Let’s explore the APIs that are defined from Cluster Resource Definitions (CRDs): 


ClusterBuildStrategy 
Represents the type of build to execute. 


Build 
Represents the build. It includes the specification of one ClusterBuildStrategy 
object. 


BuildRun 
Represents a running build. The build starts when this object is created. 


Run the following command to check all available ClusterBuildStrategy (CBS) 
objects: 


kubectl get cbs 


You should get a list of available CBSs to consume: 


NAME AGE 
buildah 26s 
buildkit 26s 
buildpacks-v3 26s 
buildpacks-v3-heroku 26s 
kaniko 26s 
kaniko-trivy 26s 
ko 26s 
source-to-image 26s 


source-to-image-redhat 26s 


This CRD is cluster-wide, available for all namespaces. If you don’t 
see any items, please install the Shipwright build strategies as dis- 
cussed previously. 


Shipwright will generate a container image on the Kubernetes nodes container cache, 
and then it can push it to a container registry. 


You need to provide the credentials to push the image to the registry in the form 
of a Kubernetes Secret. For example, if you use Quay you can create one like the 
following: 

REGISTRY_SERVER=quay. io 


REGISTRY_USER=<your_registry_user> 
REGISTRY_PASSWORD=<your_registry_password> 
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Now lets create a build-kaniko.yaml file containing the Build object that will use 
kaniko to containerize a Node.js sample app. You can find the source code in this 


EMAIL=<your_email> 

kubectl create secret docker-registry push-secret \ 
--docker-server=SREGISTRY_SERVER \ 
--docker-username=$REGISTRY_USER \ 
--docker-password=$REGISTRY_PASSWORD \ 
--docker-email=$EMAIL 


With Quay, you can use an encrypted password instead of using 
your account password. See the documentation for more details. 


book's repository: 


o © 8 8 


© 


apiVersion: shipwright.io/vialpha1 
kind: Build 
metadata: 
name: buildpack-nodejs-build 
spec: 
source: 
url: https://github.com/shipwright-io/sample-nodejs (1) 
contextDir: docker-build @ 
strategy: 
name: kaniko @ 
kind: ClusterBuildStrategy 
output: 
image: quay.io/gitops-cookbook/sample-nodejs:latest (4) 
credentials: 
name: push-secret (5] 


Repository to grab the source code from. 
The directory where the source code is present. 


The ClusterBuildStrategy to use. 


The destination of the resulting container image. Change this with your con- 


tainer registry repo. 


The secret to use to authenticate to the container registry and push the image. 
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Now, let’s create the Build object: 
kubectl create -f build-kaniko. yaml 
You should get output similar to this: 
build.shipwright.io/kaniko-nodejs-build created 
Let's list the available builds: 
kubectl get builds 


You should get output similar to the following: 


NAME REGISTERED REASON BUILDSTRATEGYKIND4S 
BUILDSTRATEGYNAME CREATIONTIME 

kaniko-nodejs-build True Succeeded ClusterBuildStrategy 
kaniko 13s 


At this point, your Build is REGISTERED, but it’s not started yet. Let’s create the 
following object in order to start it: 


apiVersion: shipwright.io/vialpha1 
kind: BuildRun 
metadata: 
generateName: kaniko-nodejs-buildrun- 
spec: 
buildRef: 
name: kaniko-nodejs-build 


kubectl create -f buildrun.yaml 


If you check the list of running pods, you should see one being created: 


kubectl get pods 


NAME READY STATUS RESTARTS b 
AGE 

kaniko-nodejs-buildrun-b9mmb-qbrgl-pod-dk7xt 0/3 PodInitializing 0b 
19s 


When the STATUS changes, the build will start, and you can track the progress by 
checking the logs from the containers used by this pod to run the build in multiple 
steps: 


step-source-default 
The first step, used to get the source code 


step-build-and-push 
The step to run the build, either from source code or from a Dockerfile like in 
this case with kaniko 


step-results 
The result of the build 
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Let’s check the logs of the building phase: 


kubectl logs -f kaniko-nodejs-buildrun-b9mmb-qbrgl-pod-dk7xt -c step-build-and-push 


INFO[0001] Retrieving image manifest ghcr.io/shipwright-io/shipwright-samples/ 
node: 12 
INFO[0001] Retrieving image ghcr.io/shipwright-io/shipwright-samples/node:124 
from registry ghcr.io 
INFO[0002] Built cross stage deps: map[] 
INFO[0002] Retrieving image manifest ghcr.io/shipwright-io/shipwright-samples/ 
node:12 
INFO[0002] Returning cached image manifest 
INFO[0002] Executing © build triggers 
INFO[0002] Unpacking rootfs as cmd COPY . /app requires it. 
INFO[0042] COPY . /app 
INFO[0042] Taking snapshot of files... 
INFO[0042] WORKDIR /app 
INFO[0042] cmd: workdir 
INFO[0042] Changed working directory to /app 
INFO[0042] No files changed in this command, skipping snapshotting. 
INFO[0042] RUN pwd && ls -l && npm install &&& 
npm run print-http-server-version 
INFO[0042] Taking snapshot of full filesystem... 
INFO[0052] cmd: /bin/sh 


INFO[0052] args: [-c pwd && ls -l && npm install &&& 
npm run print-http-server-version] 

INFO[0052] Running: [/bin/sh -c pwd && ls -l && npm install &&& 
npm run print-http-server-version] 

/app 

total 44 


-rw-r--r-- 1 node node 261 Jan 27 14:29 Dockerfile 
-rw-r--r-- 1 node node 30000 Jan 27 14:29 package-lock. json 
-fw-r--r-- 1 node node 267 Jan 27 14:29 package. json 
drwxr-xr-x 2 node node 4096 Jan 27 14:29 public 

npm WARN npm-simple-renamed@0.0.1 No repository field. 

npm WARN npm-simple-renamed@0.0.1 No license field. 


added 90 packages from 40 contributors and audited 90 packages in 6.405s 


10 packages are looking for funding 
run ‘npm fund’ for details 


found © vulnerabilities 


> npm-simple-renamed@0.0.1 print-http-server-version /app 
> serve -v 


13.0.2 

INFO[0060] Taking snapshot of full filesystem... 
INFO[0062] EXPOSE 8080 

INFO[0062] cmd: EXPOSE 

INFO[0062] Adding exposed port: 8080/tcp 
INFO[0062] CMD ["npm", "start"] 
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INFO[0070] Pushing image to quay.io/gitops-cookbook/sample-nodejs: latest 
INFO[0393] Pushed image to 1 destinations 


The image is built and pushed to the registry, and you can check the result from this 
command as well: 


kubectl get buildruns 


And on your registry, as shown in Figure 3-4. 


@RED HAT Quay.io EXPLORE APPLICATIONS REPOSITORIES TUTORIAL search a +- ya) E] dvesma.. 


Quay.lo now supports Red Hat Single Si 


€ Repositories 


(i) Repository Tags E SI 
LSS hdi 1-1of1 Filter Tags. 
TAG LAST MODIFIED | SECURITY SCAN SIZE EXPIRES MANIFEST 
5 latest 10 minutes ago 337.7MB Never SHAZS6 scosc3ibafab LE 
J 


Figure 3-4. Image pushed to Quay 


Discussion 


Shipwright provides a convenient way to create container images on Kubernetes, and 
its agnostic approach makes it robust and interoperable. The project aims at being 
the Build API for Kubernetes, providing an easier path for developers to automate on 
Kubernetes. As Tekton runs under the hood creating builds, Shipwright also makes 
transitioning from micropipeline to extended pipeline workflows on Kubernetes 
easier. 


As a reference, if you would like to create a build with Buildah instead of kaniko, it’s 
just a ClusterBuildStrategy change in your Build object: 


apiVersion: shipwright.io/vialpha1 
kind: Build 
metadata: 
name: buildpack-nodejs-build 
spec: 
source: 
url: https://github.com/shipwright-io/sample-nodejs 
contextDir: source-build 
strategy: 
name: buildah @ 
kind: ClusterBuildStrategy 
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output: 
image: quay.io/gitops-cookbook/sample-nodejs: latest 
credentials: 
name: push-secret 


@ As we discussed previously in Recipe 3.3, Buildah can create the container image 
from the source code. It doesnt need a Dockerfile. 


@ Selecting Buildah as the ClusterBuildStrategy. 


3.6 Final Thoughts 


The container format is the de facto standard for packaging applications, and today 
many tools help create container images. Developers can create images with Docker 
or with other tools and frameworks and then use the same with any CI/CD system to 
deploy their apps to Kubernetes. 


While Kubernetes per se doesn’t build container images, some tools interact with 
the Kubernetes API ecosystem to add this functionality. This aspect improves devel- 
opment velocity and consistency across environments, delegating this complexity to 
the platform. 


In the following chapters, you will see how to control the deployment of your con- 
tainers running on Kubernetes with tools such as Kustomize or Helm, and then how 
to add automation to support highly scalable workloads with CI/CD and GitOps. 
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CHAPTER 4 
Kustomize 


Deploying to a Kubernetes cluster is, in summary, applying some YAML files and 
checking the result. 


The hard part is developing the initial YAML files version; after that, usually, they 
suffer only small changes such as updating the container image tag version, the num- 
ber of replicas, or a new configuration value. One option is to make these changes 
directly in the YAML files—it works, but any error in this version (modification of 
the wrong line, deleting something by mistake, putting in the wrong whitespace) 
might be catastrophic. 


For this reason, some tools let you define base Kubernetes manifests (which change 
infrequently) and specific files (maybe one for each environment) for setting the 
parameters that change more frequently. One of these tools is Kustomize. 


In this chapter, you'll learn how to use Kustomize to manage Kubernetes resource 
files in a template-free way without using any DSL. 


The first step is to create a Kustomize project and deploy it to a Kubernetes cluster 
(see Recipe 4.1). 


After the first deployment, the application is automatically updated with a new 
container image, a new configuration value, or any other field, such as the replica 
number (see Recipes 4.2 and 4.3). 


If you've got several running environments (i.e., staging, production, etc.), you need 
to manage them similarly. Still, with its particularities, Kustomize lets you define a set 
of custom values per environment (see Recipe 4.4). 


Application configuration values are properties usually mapped as a Kubernetes 
ConfigMap. Any change (and its consequent update on the cluster) on a ConfigMap 


43 


doesn't trigger a rolling update of the application, which means that the application 
will run with the previous version until you manually restart it. 


Kustomize provides some functions to automatically execute a rolling update when 
the ConfigMap of an application changes (see Recipe 4.5). 


4.1 Using Kustomize to Deploy Kubernetes Resources 


Problem 


You want to deploy several Kubernetes resources at once. 


Solution 
Use Kustomize to configure which resources to deploy. 


Deploying an application to a Kubernetes cluster isn't as trivial as just applying one 
YAML/JSON file containing a Kubernetes Deployment object. Usually, other Kuber- 
netes objects must be defined like Service, Ingress, ConfigMaps, etc., which makes 
things a bit more complicated in terms of managing and updating these resources 
(the more resources to maintain, the more chance to update the wrong one) as well as 
applying them to a cluster (should we run multiple kubectl commands’). 


Kustomize is a CLI tool, integrated within the kubectl tool to manage, customize, 
and apply Kubernetes resources in a template-less way. 


With Kustomize, you need to set a base directory with standard Kubernetes resource 
files (no placeholders are required) and create a kustomization.yaml file where 
resources and customizations are declared, as you can see in Figure 4-1. 


. apiVersion: kustomize.config.k85.i0/vibetal 
|-kustomization.yaml —> kind: Kustomization 


|- namespace. yaml —_—= G 
de Tovment -vant . /namespace.yaml 
ii . 
k oy y registers 7 - /deployment. yaml 
L service. yanl <———_—_ ./service.yaml 


Figure 4-1. Kustomize layout 


Lets deploy a simple web page with HTML, JavaScript, and CSS files. 


First, open a terminal window and create a directory named pacman, then create 
three Kubernetes resource files to create a Namespace, a Deployment, and a Service 
with the following content. 
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The namespace at pacman/namespace. yam: 


apiVersion: v1 
kind: Namespace 
metadata: 

Name: pacman 


The deployment file at pacman/deployment.yamI: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
name: pacman-kikd 
Namespace: pacman 
labels: 
app.kubernetes.io/name: pacman-kikd 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman-kikd 
template: 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
spec: 
containers: 
- image: lordofthejars/pacman-kikd:1.0.0 
imagePullPolicy: Always 
name: pacman-kikd 
ports: 
- containerPort: 8080 
name: http 
protocol: TCP 


The service file at pacman/service.yaml: 


apiVersion: vi 
kind: Service 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
name: pacman-kikd 
Namespace: pacman 
spec: 
ports: 
- name: http 
port: 8080 
targetPort: 8080 
selector: 
app.kubernetes.io/name: pacman-kikd 


Notice that these files are Kubernetes files that you could apply to a Kubernetes 
cluster without any problem as no special characters or placeholders are used. 
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The second thing is to create the kustomization.yaml file in the pacman directory 
containing the list of resources that belongs to the application and are applied when 
running Kustomize: 


apiVersion: kustomize.config.k8s.io/vibetal 
kind: Kustomization @ 

resources: @ 

- ./mamespace.yaml 

- ./deployment. yaml 

- ./service.yaml 


@ Kustomization file 


© Resources belonging to the application processed in depth-first order 


At this point, we can apply the kustomization file into a running cluster by running 
the following command: 


kubectl apply --dry-run=client -o yaml \ 1) 
-k ./ 00 


@ Prints the result of the kustomization run, without sending the result to the 
cluster 


With -k option sets kubectl to use the kustomization file 


Directory with parent kustomization.yaml file 


We assume you've already started a Minikube cluster as shown in 
Recipe 2.3. 


The output is the YAML file that would be sent to the server if the dry-run option 
was not used: 


apiVersion: v1 
items: © 
- apiVersion: v1 
kind: Namespace (2) 
metadata: 
name: pacman 
- apiVersion: v1 
kind: Service © 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
name: pacman-kikd 
namespace: pacman 
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spec: 
ports: 
- name: http 
port: 8080 
targetPort: 8080 
selector: 
app.kubernetes.io/name: pacman-kikd 
apiVersion: apps/v1 
kind: Deployment (4) 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
name: pacman-kikd 
namespace: pacman 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman-kikd 
template: 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
spec: 
containers: 
- image: lordofthejars/pacman-kikd:1.0.0 
imagePullPolicy: Always 
name: pacman-kikd 
ports: 
- containerPort: 8080 
name: http 
protocol: TCP 


kind: List 
metadata: {} 


List of all Kubernetes objects defined in kustomization.yaml to apply 
The namespace document 


The service document 
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The deployment document 


Discussion 


The resources section supports different inputs in addition to directly setting the 
YAML files. 


For example, you can set a base directory with its own kustomization.yaml and 
Kubernetes resources files and refer it from another kustomization.yaml file placed in 
another directory. 
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Given the following directory layout: 


}— base 

| ļ— kustomization. yaml 
| L— deployment. yaml 
ļ— kustomization. yaml 

ļ— configmap. yaml 


And the Kustomization definitions in the base directory: 


apiVersion: kustomize.config.k8s.io/vibetal 

kind: Kustomization 

resources: 

- ./deployment.yaml 
You'll see that the root directory has a link to the base directory and a ConfigMap 
definition: 

apiVersion: kustomize.config.k8s.io/vibetal 

kind: Kustomization 

resources: 

- ./base 

- ./configmap.yaml 
So, applying the root kustomization file will automatically apply the resources defined 
in the base kustomization file. 


Also, resources can reference external assets from a URL following the HashiCorp 
URL format. For example, we refer to a GitHub repository by setting the URL: 


resources: 
- github.com/lordofthejars/mysql 1) 
- github.com/lordofthejars/mysql?ref=test (2) 


@ Repository with a root-level kustomization.yaml file 


© Repository with a root-level kustomization.yaml file on branch test 


You've seen the application of a Kustomize file using kubectl, but Kustomize also 
comes with its own CLI tool offering a set of commands to interact with Kustomize 
resources. 


The equivalent command to build Kustomize resources using kustomize instead of 
kubectl is: 


kustomize build 
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And the output is: 


apiVersion: v1 
kind: Namespace 
metadata: 
name: pacman 
apiVersion: v1 
kind: Service 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
name: pacman-kikd 
Namespace: pacman 
spec: 
ports: 
- name: http 
port: 8080 
targetPort: 8080 
selector: 
app.kubernetes.io/name: pacman-kikd 
apiVersion: apps/v1 
kind: Deployment 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
name: pacman-kikd 
Namespace: pacman 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman-kikd 
template: 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
spec: 
containers: 
- image: lordofthejars/pacman-kikd:1.0.0 
imagePullPolicy: Always 
name: pacman-kikd 


ports: 
- containerPort: 8080 
name: http 


protocol: TCP 


If you want to apply this output generated by kustomize to the cluster, run the 
following command: 


kustomize build . | kubectl apply -f - 
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See Also 


e Kustomize 
e kustomize/v4.4.1 on GitHub 
e HashiCorp URL format 


4.2 Updating the Container Image in Kustomize 


Problem 


You want to update the container image from a deployment file using Kustomize. 


Solution 
Use the images section to update the container image. 


One of the most important and most-used operations in software development 
is updating the application to a newer version either with a bug fix or with a 
new feature. In Kubernetes, this means that you need to create a new container 
image, and name it accordingly using the tag section (<registry>/<username>/ 
<project>:<tag>). 


Given the following partial deployment file: 


spec: 
containers: 
- image: lordofthejars/pacman-kikd:1.0.0 1) 
imagePullPolicy: Always 
Name: pacman-kikd 


@ Service 1.0.0 is deployed 


We can update the version tag to 1.0.1 by using the images section in the kustomiza- 
tion.yaml file: 


apiVersion: kustomize.config.k8s.io/vibetal 

kind: Kustomization 

resources: 

- ./nmamespace.yaml 

- ./deployment. yaml 

- ./service.yaml 

images: 

- name: lordofthejars/pacman-kikd (2) 
newTag: 1.0.1 © 


50 | Chapter 4: Kustomize 


@ images section 
© Sets the name of the image to update 


© Sets the new tag value for the image 


Finally, use kubectl in dry-run or kustomize to validate that the output of the 
deployment file contains the new tag version. In a terminal window, run the follow- 
ing command: 


kustomize build 


The output of the preceding command is: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
Name: pacman-kikd 
Namespace: pacman 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman-kikd 
template: 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
spec: 
containers: 
- image: lordofthejars/pacman-kikd:1.0.1 (1) 
imagePullPolicy: Always 
name: pacman-kikd 
ports: 
- containerPort: 8080 
name: http 
protocol: TCP 


@ Version set in the kustomize. yaml file 


Kustomize is not intrusive, which means that the original deploy- 
ment.yaml file still contains the original tag (1.0.0). 
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Discussion 


One way to update the newTag field is by editing the kustomization.yaml file, but you 
can also use the kustomize tool for this purpose. 


Run the following command in the same directory as the kustomization.yaml file: 
kustomize edit set image lordofthejars/pacman-kikd:1.0.2 


Check the content of the kustomization.yaml file to see that the newTag field has been 
updated: 


images: 
- name: Lordofthejars/pacman-kikd 
newTag: 1.0.2 


4.3 Updating Any Kubernetes Field in Kustomize 


Problem 


You want to update a field (i.e., number of replicas) using Kustomize. 


Solution 
Use the patches section to specify a change using the JSON Patch specification. 


In the previous recipe, you saw how to update the container image tag, but sometimes 
you might change other parameters like the number of replicas or add annotations, 
labels, limits, etc. 


To cover these scenarios, Kustomize supports the use of JSON Patch to modify any 
Kubernetes resource defined as a Kustomize resource. To use it, you need to specify 
the JSON Patch expression to apply and which resource to apply the patch to. 


For example, we can modify the number of replicas in the following partial deploy- 
ment file from one to three: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
Name: pacman-kikd 
Namespace: pacman 
labels: 
app.kubernetes.io/name: pacman-kikd 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman-kikd 
template: 
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metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
spec: 
containers: 


First, let’s update the kustomization.yaml file to modify the number of replicas defined 
in the deployment file: 


© 
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apiVersion: kustomize.config.k8s.io/vibetal 
kind: Kustomization 
resources: 
- ./deployment.yaml 
patches: 
- target: O 
version: v1 
group: apps 
kind: Deployment 
name: pacman-kikd 
Namespace: pacman 
patch: |- © 
- op: replace (4) 
path: /spec/replicas (5) 
value: 3 


Patch resource. 


target section sets which Kubernetes object needs to be changed. These values 
match the deployment file created previously. 


Patch expression. 
Modification of a value. 
Path to the field to modify. 


New value. 


Finally, use kubectl in dry-run or kustomize to validate that the output of the 
deployment file contains the new tag version. In a terminal window, run the follow- 
ing command: 


kustomize build 


The output of the preceding command is: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
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Name: pacman-kikd 
Namespace: pacman 
spec: 
replicas: 3 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman-kikd 


The replicas value can also be updated using the replicas field in 
the kustomization.yaml file. 


The equivalent Kustomize file using the replicas field is shown in 
the following snippet: 

apiVersion: kustomize.config.k8s.io/vibeta1 

kind: Kustomization 

replicas: 

- Mame: pacman-kikd 1] 

count: 3 O 
resources: 
- deployment. yaml 


@ Deployment to update the replicas 


@ New replicas value 


Kustomize lets you add (or delete) values, in addition to modifying a value. Let’s see 
how to add a new label: 


patches: 


- target: 
version: v1 
group: apps 


kind: Deployment 

name: pacman-kikd 

Namespace: pacman 

patch: |- 

- op: replace 
path: /spec/replicas 
value: 3 

- op: add (1) 
path: /metadata/labels/testkey (2) 
value: testvalue 


@ Adds a new field with value 
@ Path with the field to add 


© The value to set 
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The result of applying the file is: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
testkey: testvalue (1) 
Name: pacman-kikd 
Namespace: pacman 
spec: 
replicas: 3 
selector: 


@ Added label 


Discussion 


Instead of embedding a JSON Patch expression, you can create a YAML file with a 
Patch expression and refer to it using the path field instead of patch. 


Create an external patch file named external_patch containing the JSON Patch 
expression: 


- op: replace 
path: /spec/replicas 
value: 3 
- op: add 
path: /metadata/labels/testkey 
value: testvalue 


And change the patch field to path pointing to the patch file: 


patches: 


- target: 
version: v1 
group: apps 


kind: Deployment 
name: pacman-kikd 
Namespace: pacman 

path: external_patch. yaml (13 


@ Path to external patch file 


In addition to the JSON Patch expression, Kustomize also supports Strategic Merge 
Patch to modify Kubernetes resources. In summary, a Strategic Merge Patch (or SMP) 
is an incomplete YAML file that is merged against a completed YAML file. 


Only a minimal deployment file with container name information is required to 
update a container image: 
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apiVersion: kustomize.config.k8s.io/vibetal 
kind: Kustomization 


resources: 
- ./deployment. yaml 
patches: 
- target: 
labelSelector: "app.kubernetes.io/name=pacman-kikd" 1] 
patch: |- (2) 


apiVersion: apps/v1 © 
kind: Deployment 
metadata: 
name: pacman-kikd 
spec: 
template: 
spec: 
containers: 
- name: pacman-kikd 
image: lordofthejars/pacman-kikd:1.2.0 (4) 


Target is selected using label 
Patch is smart enough to detect if it is an SMP or JSON Patch 


This is a minimal deployment file 


© © 8 86 


Sets only the field to change, the rest is left as is 


The generated output is the original deployment.yaml file but with the new container 
image: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
name: pacman-kikd 
namespace: pacman 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman-kikd 
template: 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
spec: 
containers: 
- image: lordofthejars/pacman-kikd:1.2.0 
imagePullPolicy: Always 
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path is supported as well. 


See Also 


e RFC 6902: JavaScript Object Notation (JSON) Patch 
e Strategic Merge Patch 


4.4 Deploying to Multiple Environments 


Problem 


You want to deploy the same application in different namespaces using Kustomize. 


Solution 
Use the namespace field to set the target namespace. 


In some circumstances, it’s good to have the application deployed in different name- 
spaces; for example, one namespace can be used as a staging environment, and 
another one as the production namespace. In both cases, the base Kubernetes files are 
the same, with minimal changes like the namespace deployed, some configuration 
parameters, or container version, to mention a few. Figure 4-2 shows an example. 


apiVersion: kustomize.config.k85.i0/vibetal 
[-kustomtzatton. yaml ——>| kind: Kustomization 


namespace. yaml aaa i 
[de Vovmant vant /namespace. yaml 
RPTP 
oe y registers ./deployment. yaml 
L_service.yaml ————_ | /service. yaml 


Figure 4-2. Kustomize layout 


kustomize lets you define multiple changes with a different namespace, as overlays 
on a common base using the namespace field. For this example, all base Kubernetes 
resources are put in the base directory and a new directory is created for customiza- 
tions of each environment: 


}— base (1) 


| į deployment.yaml 
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| ‘  kustomization.yaml 
ļ— production 
| L— kustomization.yaml (2) 
L— staging 
L— kustomization.yaml © 


@ Base files 
@ Changes specific to production environment 


© Changes specific to staging environment 
The base kustomization file contains a reference to its resources: 


apiVersion: kustomize.config.k8s.io/vibetal 
kind: Kustomization 

resources: 

- ./deployment. yaml 


There is a kustomization file with some parameters set for each environment direc- 
tory. These reference the base directory, the namespace to inject into Kubernetes 
resources, and finally, the image to deploy, which in production is 1.1.0 but in staging 
is 1.2.0-beta. 


For the staging environment, kustomization.yaml content is shown in the following 
listing: 


apiVersion: kustomize.config.k8s.io/vibeta1 
kind: Kustomization 
resources: 
- ../base (13 
namespace: staging (2) 
images: 
- name: lordofthejars/pacman-kikd 
newTag: 1.2.0-beta © 


@ References to base directory 
@ Sets namespace to staging 


© Sets the container tag for the staging environment 


The kustomization file for production is similar to the staging one, but changes the 
namespace and the tag: 


apiVersion: kustomize.config.k8s.io/vibetal 
kind: Kustomization 

resources: 

- ../base 

Namespace: prod (1) 

images: 
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- name: lordofthejars/pacman-kikd 
newTag: 1.1.0 (2) 


@ Sets namespace for production 


© Sets the container tag for the production environment 


Running kustomize produces different output depending on the directory where it is 
run; for example, running kustomize build in the staging directory produces: 


o 
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apiVersion: apps/v1 
kind: Deployment 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
name: pacman-kikd 
namespace: staging (1) 
spec: 
replicas: 1 


template: 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
spec: 
containers: 
- image: lordofthejars/pacman-kikd:1.2.0-beta (2) 


Namespace value is injected 


Container tag for the staging environment is injected 


But if you run it in the production directory, the output is adapted to the production 
configuration: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
name: pacman-kikd 
namespace: prod (13 
spec: 
replicas: 1 


spec: 


containers: 
- image: lordofthejars/pacman-kikd:1.1.0 (2) 


Injects the production namespace 
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© Container tag for the production environment 


Discussion 


Kustomize can preappend/append a value to the names of all resources and refer- 
ences. This is useful when a different name in the resource is required depending on 
the environment, or to set the version deployed in the name: 


apiVersion: kustomize.config.k8s.io/vibeta1 
kind: Kustomization 
resources: 
- ../base 
Namespace: staging 
namePrefix: staging- 
nameSuffix: -vi-2-0 @ 
images: 
- name: Lordofthejars/pacman-kikd 
NewTag: 1.2.0-beta 


@ Prefix to preappend 


© Suffix to append 
And the resulting output is as follows: 
apiVersion: apps/v1 
kind: Deployment 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
Name: staging-pacman-kikd-v1-2-0 (13 
namespace: staging 
spec: 


© New name of the deployment file 


4.5 Generating ConfigMaps in Kustomize 


Problem 


You want to generate Kubernetes ConfigMaps using Kustomize. 


Solution 


Use the ConfigMapGenerator feature field to generate a Kubernetes ConfigMap 
resource on the fly. 
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Kustomize provides two ways of adding a ConfigMap as a Kustomize resource: either 
by declaring a ConfigMap as any other resource or declaring a ConfigMap from a 
ConfigMapGenerator. 


While using ConfigMap as a resource offers no other advantage than populating 
Kubernetes resources as any other resource, ConfigMapGenerator automatically 
appends a hash to the ConfigMap metadata name and also modifies the deployment 
file with the new hash. This minimal change has a deep impact on the application's 
lifecycle, as we'll see soon in the example. 


Let’s consider an application running in Kubernetes and configured using a Config 
Map—for example, a database timeout connection parameter. We decided to increase 
this number at some point, so the ConfigMap file is changed to this new value, and we 
deploy the application again. Since the ConfigMap is the only changed file, no rolling 
update of the application is done. A manual rolling update of the application needs 
to be triggered to propagate the change to the application. Figure 4-3 shows what is 
changed when a ConfigMap object is updated. 


Kubernetes Cluster 


O9 © OG 


1. Initial version 2. Changeon ConfigMap 3. Only ConfigMap is updated 


Figure 4-3. Change of a ConfigMap 


But, if ConfigMapGenerator manages the ConfigMap, any change on the configura- 
tion file also changes the deployment Kubernetes resource. Since the deployment file 
has changed too, an automatic rolling update is triggered when the resources are 
applied, as shown in Figure 4-4. 


Moreover, when using ConfigMapGenerator, multiple configuration datafiles can be 
combined into a single ConfigMap, making a perfect use case when every environ- 
ment has different configuration files. 
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Kubernetes Cluster 


C 


deploy 


1. Initial version 2. Change on 3. ConfigMapGenerator 
ConfigMap updates both 


Figure 4-4. Change of a ConfigMap using ConfigMapGenerator 


Let's start with a simple example, adding the ConfigMapGenerator section in the 
kustomization.yaml file. 


The deployment file is similar to the one used in previous sections of this chapter but 
includes the volumes section: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
Name: pacman-kikd 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman-kikd 
template: 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
spec: 
containers: 
- image: lordofthejars/pacman-kikd:1.0.0 
imagePullPolicy: Always 
Name: pacman-kikd 
volumeMounts: 
- name: config 
mountPath: /config 
volumes: 
- name: config 
configMap: 
name: pacman-configmap @ 


@ ConfigMap name is used in the kustomization.yaml file 
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The configuration properties are embedded within the kustomization.yaml file. 
Notice that the ConfigMap object is created on the fly when the kustomization file 
is built: 


apiVersion: kustomize.config.k8s.io/vibeta1 
kind: Kustomization 
resources: 
- ./deployment.yaml 
configMapGenerator: 
- name: pacman-configmap (1) 
literals: 
- db-timeout=2000 © 
- db-username=Ada 


@ Name of the ConfigMap set in the deployment file 
@ Embeds configuration values in the file 


© Sets a key/value pair for the properties 


Finally, use kubectl in dry-run or kustomize to validate that the output of the 
deployment file contains the new tag version. In a terminal window, run the follow- 
ing command: 


kustomize build 


The output of the preceding command is a new ConfigMap with the configuration 
values set in kustomization.yaml. Moreover, the name of the ConfigMap is updated by 
appending a hash in both the generated ConfigMap and deployment: 


apiVersion: v1 
data: @ 
db-timeout: "2000" 
db-username: Ada 
kind: ConfigMap 
metadata: 
name: pacman-configmap-96kb69b6t4 e 
apiVersion: apps/v1 
kind: Deployment 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
Name: pacman-kikd 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman-kikd 
template: 
metadata: 
labels: 
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app.kubernetes.io/name: pacman-kikd 
spec: 
containers: 
- image: lordofthejars/pacman-kikd:1.0.0 
imagePullPolicy: Always 
Name: pacman-kikd 
volumeMounts: 
- mountPath: /config 
name: config 
volumes: 
- configMap: 
name: pacman-configmap-96kb69b6t4 © 
name: config 


@ ConfigMap with properties 
© Name with hash 


© Name field is updated to the one with the hash triggering a rolling update 


Since the hash is calculated for any change in the configuration properties, a change 
on them provokes a change on the output triggering a rolling update of the applica- 
tion. Open the kustomization.yaml file and update the db-timeout literal from 2000 
to 1000 and run kustomize build again. Notice the change in the ConfigMap name 
using a new hashed value: 


apiVersion: v1 
data: 
db-timeout: "1000" 
db-username: Ada 
kind: ConfigMap 
metadata: 
name: pacman-configmap-6952t58tb4 1] 
apiVersion: apps/v1 
kind: Deployment 


volumes: 
- configMap: 


name: pacman-configmap-6952t58tb4 
name: config 


@ New hashed value 


Discussion 


ConfigMapGenerator also supports merging configuration properties from different 
sources. 
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Create a new kustomization.yaml file in the dev_literals directory, setting it as the 
previous directory and overriding the db- username value: 


apiVersion: kustomize.config.k8s.io/vibetal 
kind: Kustomization 
resources: 
- ../literals (1) 
configMapGenerator: 
- name: pacman-configmap 
behavior: merge 
literals: 
- db-username=Alexandra © 


@ Base directory 
© Merge properties (can be create or replace too) 


©  Overridden value 


Running the kustomize build command produces a ConfigMap containing a merge 
of both configuration properties: 


apiVersion: v1 

data: 
db-timeout: "1000" @ 
db-username: Alexandra @ 

kind: ConfigMap 

metadata: 
name: pacman-configmap-ttfdfdk5t8 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
Name: pacman-kikd 


@ Inherits from base 


© Overrides value 


In addition to setting configuration properties as literals, Kustomize supports defin- 
ing them as .properties files. 


Create a connection.properties file with two properties inside: 


db-url=prod:4321/db 
db-username=ada 


The kustomization.yaml file uses the files field instead of literals: 
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apiVersion: kustomize.config.k8s.io/vibetal 
kind: Kustomization 
resources: 
- ./deployment. yaml 
configMapGenerator: 
- name: pacman-configmap 
files: © 
- ./connection.properties (2) 


@ Sets a list of files to read 


@ Path to the properties file 


Running the kustomize build command produces a ConfigMap containing the name 
of the file as a key, and the value as the content of the file: 


apiVersion: vi 
data: 
connection.properties: |- 
db-url=prod:4321/db 
db-username=ada 
kind: ConfigMap 
metadata: 
Name: pacman-configmap-g9dm2gtt77 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
labels: 
app.kubernetes.io/name: pacman-kikd 
Name: pacman-kikd 


See Also 


Kustomize offers a similar way to deal with Kubernetes Secrets. But as we'll see in 
Chapter 8, the best way to deal with Kubernetes Secrets is using Sealed Secrets. 


4.6 Final Thoughts 


Kustomize is a simple tool, using template-less technology that allows you to define 
plain YAML files and override values either using a merge strategy or using JSON 
Patch expressions. The structure of a project is free as you define the directory 
layout you feel most comfortable with; the only requirement is the presence of a 
kustomization.yamI file. 


But there is another well-known tool to manage Kubernetes resources files, that in 
our opinion, is a bit more complicated but more powerful, especially when the appli- 
cation/service to deploy has several dependencies such as databases, mail servers, 
caches, etc. This tool is Helm, and we'll cover it in Chapter 5. 
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CHAPTER 5 
Helm 


In Chapter 4, you learned about Kustomize, a simple yet powerful tool to manage 
Kubernetes resources. But another popular tool aims to simplify the Kubernetes 
resources management too: Helm. 


Helm works similarly to Kustomize, but it’s a template solution and acts more like a 
package manager, producing artifacts that are versionable, sharable, or deployable. 


In this chapter, we'll introduce Helm, a package manager for Kubernetes that helps 
install and manage Kubernetes applications using the Go template language in YAML 
files. 


The first step is to create a Helm project and deploy it to a Kubernetes cluster (see 
Recipes 5.1 and 5.2). After the first deployment, the application is updated with a new 
container image, a new configuration value, or any other field, such as the replica 
number (see Recipe 5.3). 


One of the differences between Kustomize and Helm is the concept of a Chart. A 
Chart is a packaged artifact that can be shared and contains multiple elements like 
dependencies on other Charts (see Recipes 5.4, 5.5, and 5.6). 


Application configuration values are properties usually mapped as a Kubernetes 
ConfigMap. Any change (and its consequent update on the cluster) on a ConfigMap 
doesnt trigger a rolling update of the application, which means that the application 
will run with the previous version until you manually restart it. 


Helm provides some functions to automatically execute a rolling update when the 
ConfigMap of an application changes (see Recipe 5.7). 
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5.1 Creating a Helm Project 


Problem 


You want to create a simple Helm project. 


Solution 
Use the Helm CLI tool to create a new project. 


In contrast to Kustomize, which can be used either within the kubectl command or 
as a standalone CLI tool, Helm needs to be downloaded and installed in your local 
machine. 


Helm is a packager for Kubernetes that bundles related manifest files and packages 
them into a single logical deployment unit: a Chart. Thus simplified, for many 
engineers, Helm makes it easy to start using Kubernetes with real applications. 


Helm Charts are useful for addressing the installation complexities and simple 
upgrades of applications. 


For this book, we use Helm 3.7.2, which you can download from GitHub and install 
in your PATH directory. 


Open a terminal and run the following commands to create a Helm Chart directory 
layout: 


mkdir pacman 
mkdir pacman/templates 


cd pacman 


Then create three files: one that defines the Chart, another representing the deploy- 
ment template using the Go template language and template functions from the Sprig 
library, and finally a file containing the default values for the Chart. 


A Chart.yaml file declares the Chart with information such as version or name. 
Create the file in the root directory: 


apiVersion: v2 

name: pacman 

description: A Helm chart for Pacman 
type: application 

version: 0.1.0 @ 


appVersion: "1.0.0" (2) 
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@ Version of the Chart. This is updated when something in the Chart definition is 
changed. 


© Version of the application. 


Lets create a deployment.yaml and a service.yaml template file to deploy the 
application. 


The deployment.yaml file templatizes the deployment’s name, the application version, 
the replica count, the container image and tag, the pull policy, the security context, 
and the port: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
name: {{ .Chart.Name}} 1] 
labels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
{{- if .Chart.AppVersion }} 
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} © 
{{- end }} 
spec: 
replicas: {{ .Values.replicaCount }} 4] 
selector: 
matchLabels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
template: 
metadata: 
labels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
spec: 
containers: 

- image: "{{ .Values.image.repository }}: 

{{ .Values.image.tag | default .Chart.AppVersion}}" 60 
imagePullPolicy: {{ .Values.image.pullPolicy }} @ 
securityContext: 

{{- toYaml .Values.securityContext | nindent 14 }} 
name: {{ .Chart.Name}} 
ports: 
- containerPort: {{ .Values.image.containerPort }} 
name: http 
protocol: TCP 


Sets the name from the Chart.yaml file 


@ Conditionally sets the version based on the presence of the appVersion in the 
Chart.yaml file 


© Sets the appVersion value but quoting the property 
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Placeholder for the replicaCount property 
Placeholder for the container image 


© Placeholder for the image tag if present and if not, defaults to the Chart.yaml 
property 


@ Sets the securityContext value as a YAML object and not as a string, indenting 
it 14 spaces 


The service.yaml file templatizes the service name and the container port: 


apiVersion: v1 
kind: Service 
metadata: 
labels: 
app.kubernetes.io/name: {{ .Chart.Name }} 
name: {{ .Chart.Name }} 
spec: 
ports: 
- name: http 
port: {{ .Values.image.containerPort }} 
targetPort: {{ .Values.image.containerPort }} 
selector: 
app.kubernetes.io/name: {{ .Chart.Name }} 


The values.yaml file contains the default values for the Chart. These values can be 
overridden at runtime, but they provide good initial values. 
Create the file in the root directory with some default values: 
image: (1) 

repository: quay.io/gitops-cookbook/pacman-kikd (2) 

tag: "1.0.0" 

pullPolicy: Always 

containerPort: 8080 


replicaCount: 1 
securityContext: {} © 


@ Defines the image section 
@ Sets the repository property 


© Empty securityContext 


Built-in properties are capitalized; for this reason, properties defined in the 
Chart.yaml file start with an uppercase letter. 
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Since the toYaml function is used for the securityContext value, the expected value 
for the securityContext property in values.yaml should be a YAML object. For 
example: 


securityContext: 
capabilities: 

drop: 

- ALL 
readOnlyRootFilesystem: true 
runAsNonRoot: true 
runAsUser: 1000 


The relationship between all elements is shown in Figure 5-1. 


apiVersion: apps/v1i 
kind: Deployment chart.yaml 
metadata: | 


apiVersion: v2 | 
name: {{_. Chart. Name} } <4 nane: pacman 


labels: description: A Helm chart for Pacman 
app.kubernetes.io/name: {{ .Chart.Name}} 
{{- if .Chart.AppVersion }} type: application 
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} | version: 0.1.0 
{{- end }} | appVersion: "1.0.0" 
spec: : < 
replicas: {{ .Values.replicaCount }} , values.yaml 
selector: replicaCount: 1 
matchLabels: | 
app.kubernetes.io/name: {{ .Chart.Name}} ‘image: 
template: aseda lordofthejars/pacman-kikd 
. tag: "1.0.0" | 
metadata: pullPolicy: Always 
labels: containerPort: 8080 
app.kubernetes.io/name: {{ .Chart.Name}} 
spec: securityContext: {} 


containers: ee 
- image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default ].Cha 
imagePullPolicy: {{ .Values.image.pullPolicy }} 


-Appversion}}" 


securityContext: 
{{- toYaml .Values.securityContext | nindent 14 }} 
name: {{ .Chart.Name}} 
ports: 
- containerPort: {{ .Values.image.containerPort }} 
name: http 
protocol: TCP 


Figure 5-1. Relationship between Helm elements 
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At this point the Helm directory layout is created and should be similar to this: 


pacman 
| Chart.yaml (13 
ļ— templates (2) 
| ļ— deployment.yaml © 
| ļ— service.yaml 
L— values.yaml (4) 


@ The Chart.yaml file is the Chart descriptor and contains metadata related to the 
Chart. 


@ The templates directory contains all template files used for installing a Chart. 
© These files are Helm template files used to deploy the application. 


© The values.yaml file contains the default values for a Chart. 


To render the Helm Chart locally to YAML, run the following command in a terminal 
window: 


helm template . 


The output is: 


apiVersion: v1 
kind: Service 
metadata: 
labels: 
app.kubernetes.io/name: pacman 
name: pacman 
spec: 
ports: 
- name: http 
port: 8080 (2) 
targetPort: 8080 
selector: 
app.kubernetes.io/name: pacman 
apiVersion: apps/v1 
kind: Deployment 
metadata: 
name: pacman 
labels: 
app.kubernetes.io/name: pacman 
app.kubernetes.io/version: "1.0.0" © 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman 
template: 
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© © © 8 8 


metadata: 
labels: 
app.kubernetes.io/name: pacman 
spec: 
containers: 

- image: "quay.io/gitops-cookbook/pacman-kikd:1.0.0" (4) 
imagePullPolicy: Always 
securityContext: (5) 

{} 
name: pacman 
ports: 
- containerPort: 8080 
name: http 
protocol: TCP 


Name is injected from Chart.yaml 

Port is set in values.yaml 

Version is taken from Chart version 
Concatenates content from two attributes 


Empty security context 


You can override any default value by using the --set parameter in Helm. Let’s 
override the replicaCount value from one (defined in values.yaml) to three: 


helm template --set replicaCount=3 . 


And the replicas value is updated: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
Name: pacman 
labels: 
app.kubernetes.io/name: pacman 
app.kubernetes.io/version: "1.0.0" 
spec: 
replicas: 3 


Discussion 


Helm is a package manager for Kubernetes, and as such, it helps you with the task of 
versioning, sharing, and upgrading Kubernetes applications. 


Lets see how to install the Helm Chart to a Kubernetes cluster and upgrade the 
application. 
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With Minikube up and running, execute the following command in a terminal 
window, and run the install command to deploy the application to the cluster: 


helm install pacman . 
The Chart is installed in the running Kubernetes instance: 


LAST DEPLOYED: Sat Jan 22 15:13:50 2022 
NAMESPACE: default 

STATUS: deployed 

REVISION: 1 

TEST SUITE: None 


Get the list of current deployed pods, Deployments, and Services to validate that the 
Helm Chart is deployed in the Kubernetes cluster: 


kubectl get pods 


NAME READY STATUS RESTARTS AGE 
pacman-7947b988-kzjbc 1/1 Running 0 60s 


kubectl get deployment 


NAME READY UP-TO-DATE AVAILABLE AGE 
pacman 1/1 1 1 4m50s 


kubectl get services 


NAME TYPE CLUSTER-IP EXTERNAL - IP PORT(S) AGE 
pacman ClusterIP 172.30.129.123 <none> 8080/TCP 9m55s 


Also, it’s possible to get history information about the deployed Helm Chart using the 
history command: 


helm history pacman 


REVISION UPDATED STATUS CHART L 
APP VERSION DESCRIPTION 

1 Sat Jan 22 15:23:22 2022 deployed Ppacman-0.1.04 
1.0.0 Install complete 


To uninstall a Chart from the cluster, run uninstall command: 


helm uninstall pacman 


release "pacman" uninstalled 


Helm is a package manager that lets you share the Chart (package) to other Charts 
as a dependency. For example, you can have a Chart defining the deployment of 
the application and another Chart as a dependency setting a database deployment. 
In this way, the installation process installs the application and the database Chart 
automatically. 


We'll learn about the packaging process and adding dependencies in a later section. 


74 | Chapter 5: Helm 


You can use the helm create <name> command to let the Helm 
tool skaffold the project. 


See Also 


e Helm 
e Go template package 


e Sprig Function Documentation 


5.2 Reusing Statements Between Templates 


Problem 


You want to reuse template statements across several files. 


Solution 
Use _helpers.tpl to define reusable statements. 


We deployed a simple application to Kubernetes using Helm in the previous recipe. 
This simple application was composed of a Kubernetes Deployment file and a Kuber- 
netes Service file where the selector field was defined with the same value. 


As a reminder: 


spec: 
replicas: {{ .Values.replicaCount }} 
selector: 
matchLabels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
template: 
metadata: 
labels: 
app.kubernetes.io/name: {{ .Chart.Name}} 


service. yaml 


selector: 
app.kubernetes.io/name: {{ .Chart.Name }} 


If you need to update this field—for example, adding a new label as a selector—you 
would need to update in three places, as shown in the previous snippets. 
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Helm lets you create a _helpers.tpl file in the templates directory defining statements 
that can be called in templates to avoid this problem. 


Let’s refactor the previous example to use the _helpers.tpl file to define the selector 
Labels. 


Create the _helpers.tpl file in the templates directory with the following content: 


o 
(27 


{{- define "pacman.selectorLabels" -}} (1) 
app.kubernetes.io/name: {{ .Chart.Name}} (2) 
{{- end }} 


Defines the statement name 


Defines the logic of the statement 


Then replace the template placeholders shown in previous snippets with a call to the 
podman.selectorLabels helper statement using the include keyword: 


(2) 


spec: 
replicas: {{ .Values.replicaCount }} 
selector: 
matchLabels: 
{{- include "pacman.selectorLabels" . | nindent 6 }} (1) 
template: 
metadata: 
labels: 
{{- include "pacman.selectorLabels" . | nindent 8 }} (2) 
spec: 
containers: 


Calls pacman . selectorLabels with indentation 


Calls pacman . selectorLabels with indentation 


To render the Helm Chart locally to YAML, run the following command in a terminal 
window: 


helm template . 


The output is: 


apiVersion: v1 
kind: Service 
metadata: 
labels: 
app.kubernetes.io/name: pacman 
name: pacman 
spec: 
ports: 
- name: http 
port: 8080 
targetPort: 8080 
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selector: 
app.kubernetes.io/name: pacman (1) 
apiVersion: apps/v1 
kind: Deployment 
metadata: 
name: pacman 
labels: 
app.kubernetes.io/name: pacman 
app.kubernetes.io/version: "1.0.0" 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman @ 
template: 
metadata: 
labels: 
app.kubernetes.io/name: pacman © 
spec: 
containers: 

- image: "quay.io/gitops-cookbook/pacman-kikd:1.0.0" 
imagePullPolicy: Always 
securityContext: 

{} 
name: pacman 
ports: 
- containerPort: 8080 
name: http 
protocol: TCP 


Selector is updated with value set in _helpers.tpl 
Selector is updated with value set in _helpers.tpl 


Selector is updated with value set in _helpers.tpl 


Discussion 


If you want to update the selector labels, the only change you need to do is an update 
to the _helpers.tpl file: 


{{- define "pacman.selectorLabels" -}} 
app.kubernetes.io/name: {{ .Chart.Name}} 
app.kubernetes.io/version: {{ .Chart.AppVersion}} (1) 
{{- end }} 


Adds a new attribute 
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To render the Helm Chart locally to YAML, run the following command in a terminal 
window: 


helm template . 


The output is: 


# Source: pacman/templates/service.yanl 
apiVersion: v1 

kind: Service 

metadata: 


selector: 
app.kubernetes.io/name: pacman 
app.kubernetes.io/version: 1.0.0 (1) 
apiVersion: apps/v1 
kind: Deployment 
metadata: 
name: pacman 
labels: 
app.kubernetes.io/name: pacman 
app.kubernetes.io/version: "1.0.0" 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman 
app.kubernetes.io/version: 1.0.0 (2) 
template: 
metadata: 
labels: 
app.kubernetes.io/name: pacman 
app.kubernetes.io/version: 1.0.0 © 
spec: 


@ Label is added 
@ Label is added 


© Label is added 


Although it’s common to use __helpers.tpl as the filename to define 
functions, you can name any file starting with __, and Helm will 
read the functions too. 
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5.3 Updating a Container Image in Helm 


Problem 


You want to update the container image from a deployment file using Helm and 
upgrade the running instance. 


Solution 
Use the upgrade command. 
With Minikube up and running, deploy version 1.0.0 of the pacman application: 


helm install pacman . 


With the first revision deployed, let’s update the container image to a new version and 
deploy it. 


You can check revision number by running the following command: 


helm history pacman 


REVISION UPDATED STATUS CHART APP VERSIONS 
DESCRIPTION 
1 Sun Jan 23 16:00:09 2022 deployed pacman-0.1.0 1.0.06 


Install complete 


To update the version, open values.yaml and update the image. tag field to the newer 
container image tag: 


image: 
repository: quay.io/gitops-cookbook/pacman-kikd 
tag: "1.1.0" 


pullPolicy: Always 
containerPort: 8080 


replicaCount: 1 
securityContext: {} 


© Updates to container tag to 1.1.0 


Then update the appVersion field of the Chart. yam file: 


apiVersion: v2 
name: pacman 
description: A Helm chart for Pacman 


type: application 


version: 0.1.0 
appVersion: "1.1.0" 1 


@ Version is updated accordingly 
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You can use appVersion as the tag instead of having two separate 
fields. Using two fields or one might depend on your use case, 
versioning strategy, and lifecycle of your software. 


After these changes, upgrade the deployment by running the following command: 
helm upgrade pacman . 
The output reflects that a new revision has been deployed: 


Release "pacman" has been upgraded. Happy Helming! 
NAME: pacman 

LAST DEPLOYED: Mon Jan 24 11:39:28 2022 

NAMESPACE: asotobue-dev 

STATUS: deployed 

REVISION: 2 @ 

TEST SUITE: None 


@ Newrevision 
The history command shows all changes between all versions: 


helm history pacman 


REVISION UPDATED STATUS CHART APP VERSIONS 
DESCRIPTION 

1 Mon Jan 24 10:22:06 2022 superseded pacman-0.1.0 1.0.05 
Install complete 

2 Mon Jan 24 11:39:28 2022 deployed pacman-0.1.0 1.1.04 


Upgrade complete 


appVersion is the application version, so every time you change 
the application version, you should update that field too. On the 
other side, version is the Chart version and should be updated 
when the definition of the Chart (i.e., templates) changes, so both 
fields are independent. 


Discussion 


Not only you can install or upgrade a version with Helm, but you can also roll back to 
a previous revision. 


In the terminal window, run the following command: 


helm rollback pacman 1 


Rollback was a success! Happy Helming! 
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Running the history command reflects this change too: 


helm history pacman 


REVISION UPDATED STATUS CHART APP VERSIONS 
DESCRIPTION 

1 Mon Jan 24 10:22:06 2022 superseded pacman-0.1.0 1.0.06 
Install complete 

2 Mon Jan 24 11:39:28 2022 superseded pacman-0.1.0 1.1.06 
Upgrade complete 

3 Mon Jan 24 12:31:58 2022 deployed Ppacman-0.1.0 1.0.06 


Rollback to 


Finally, Helm offers a way to override values, not only using the --set argument as 
shown in Recipe 5.1, but by providing a YAML file. 


Create a new YAML file named newvalues.yaml in the root directory with the follow- 
ing content: 


image: 
tag: "1.2.0" 


Then run the template command, setting the new file as an override of values.yaml: 
helm template pacman -f newvalues.yaml . 


The resulting YAML document is using the values set in values.yaml but overriding 
the images. tag set in newvalues.yaml: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
Name: pacman 
spec: 
replicas: 1 
selector: 
matchLabels: 
app.kubernetes.io/name: pacman 
template: 
metadata: 
labels: 
app.kubernetes.io/name: pacman 
spec: 
containers: 
- image: "quay.io/gitops-cookbook/pacman-kikd:1.2.0" 
imagePullPolicy: Always 
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5.4 Packaging and Distributing a Helm Chart 


Problem 


You want to package and distribute a Helm Chart so it can be reused by others. 


Solution 
Use the package command. 


Helm is a package manager for Kubernetes. As we've seen in this chapter, the basic 
unit in Helm is a Chart containing the Kubernetes files required to deploy the 
application, the default values for the templates, etc. 


But weve not yet seen how to package Helm Charts and distribute them to be 
available to other Charts as dependencies or deployed by other users. 


Lets package the pacman Chart into a .tgz file. In the pacman directory, run the 
following command: 


helm package . 
And you'll get a message informing you where the archive is stored: 


Successfully packaged chart and saved it to: 
gitops-cookbook/code/05_helm/04_package/pacman/pacman-0.1.0.tgz 


A Chart then needs to be published into a Chart repository. A Chart repository is 
an HTTP server with an index.yaml file containing metadata information regarding 
Charts and .tgz Charts. 


To publish them, update the index.yaml file with the new metadata information, and 
upload the artifact. 


The directory layout of a repository might look like this: 


repo 
ļ— index. yaml 
ļ— pacman-0.1.0.tgz 


The index.yaml file with information about each Chart present in the repository looks 
like: 


apiVersion: v1 
entries: 
pacman: 
- apiVersion: v2 
appVersion: 1.0.0 
created: "2022-01-24T16:42:54.080959+01:00" 
description: A Helm chart for Pacman 
digest: aa3cce809ffcca86172Fc793d7804d1c61f157b9b247680a67d5b16b18a0798d 
name: pacman 
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type: application 
urls: 
- pacman-0.1.0.tgz 
version: 0.1.0 
generated: "2022-01-24T16:42:54.080485+01:00" 


You can run helm repo index in the root directory, where pack- 
aged Charts are stored, to generate the index file automatically. 


Discussion 


In addition to packaging a Helm Chart, Helm can generate a signature file for the 
packaged Chart to verify its correctness later. 


In this way, you can be sure it has not been modified, and it’s the correct Chart. 


To sign/verify the package, you need a pair of GPG keys in the machine; were 
assuming you already have one pair created. 


Now you need to call the package command but set the -sign argument with the 
required parameters to generate a signature file: 
helm package --sign --key 'me@example.com' \ 
--keyring /home/me/.gnupg/secring.gpg 
Now, two files are created—the packaged Helm Chart (.tgz) and the signature file 
(.tgz.prov): 


I~ Chart. yaml 

ļ— pacman-0.1.0.tgz (1) 

ļ— pacman-0.1.0.tgz.prov (2) 
ļ— templates 

| į deployment.yaml 

| — service.yaml 

L— values.yaml 


@ Chart package 


@ Signature file 


Remember to upload both files in the Chart repository. 
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To verify that a Chart is valid and has not been manipulated, use the verify 
command: 


helm verify pacman-0.1.0.tgz 
Signed by: alexs (book) <asotobu@example.com> 
Using Key With Fingerprint: 57C4511D738BCOB288FAF9D69B40EB787040F 3CF 


Chart Hash Verified: 
sha256:d8b2e0c5e12a8425df2ea3a903807b93aabe4a6ff8277511a7865c847de3cobf O 


@ It’s valid 


See Also 


e The Chart Repository Guide 


e Helm Provenance and Integrity 


5.5 Deploying a Chart from a Repository 


Problem 


You want to deploy a Helm Chart stored in Chart repository. 


Solution 


Use the repo add command to add the remote repository and the install command 
to deploy it. 


Public Helm Chart repositories like Bitnami are available for this purpose. 


To install Charts from a repository (either public or private), you need to register it 
using its URL: 


helm repo add bitnami https://charts.bitnami.com/bitnami 1] 
© URL of Helm Chart repository where index.yaml is placed 


List the registered repositories: 


helm repo list 


NAME URL 
stable https://charts.helm.sh/stable 
bitnami https://charts.bitnami.com/bitnami (1) 


@ Bitnami repo is registered 
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Run helm repo update to get the latest list of Charts for each repo. 


After registering a repository, you might want to find which Charts are available. 
If you want to deploy a PostgreSQL instance in the cluster, use the search command 
to search all repositories for a Chart that matches the name: 

helm search repo postgresql 


The outputs are the list of Charts that matches the name, the version of the Chart 
and PostgreSQL, and a description. Notice the name of the Chart is composed of the 
repository name and the Chart name, i.e., bitnami/postgresqlL: 


NAME CHART VERSION APP VERSIONS 
DESCRIPTION 
bitnami/postgresql 10.16.2 11.14.06 
Chart for PostgreSQL, an object-relational data... 
bitnami/postgresql-ha 8.2.6 11.14.06 
Chart for PostgreSQL with HA architecture (usin... 
stable/postgresql 8.6.4 11.7.05 
DEPRECATED Chart for PostgreSQL, an object-rela... 
stable/pgadmin 1.2.2 4.18.06 
pgAdmin is a web based administration tool for ... 
stable/stolon 1.6.5 0.16.04 
DEPRECATED - Stolon - PostgreSQL cloud native H... 
stable/gcloud-sqlproxy 0.6.1 1.115 
DEPRECATED Google Cloud SQL Proxy 
stable/prometheus-postgres-exporter 1.3.1 0.8.04 


DEPRECATED A Helm chart for prometheus postgres... 


To deploy the PostgreSQL Chart, run the install command but change the location 
of the Helm Chart from a local directory to the full name of the Chart (<repo>/ 
<chart>): 

helm install my-db \ (1) 

--set postgresql.postgresqlUsername=my-default,postgresql. t 

postgresqlPassword=postgres, postgresql.postgresqlDatabase=mydb, t> 


postgresql.persistence.enabled=false \ (2) 
bitnami/postgresql © 


@ Sets the name of the deployment 
© Overrides default values to the ones set in the command line 


© Sets the PostgreSQL Chart stored in the Bitnami repo 
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And a detailed output is shown in the console: 


NAME: my-db 

LAST DEPLOYED: Mon Jan 24 22:33:56 2022 
NAMESPACE: asotobue-dev 

STATUS: deployed 

REVISION: 1 

TEST SUITE: None 

NOTES: 

CHART NAME: postgresql 

CHART VERSION: 10.16.2 

APP VERSION: 11.14.0 


** Please be patient while the chart is being deployed ** 


PostgreSQL can be accessed via port 5432 on the following DNS namest 
from within your cluster: 


my-db-postgresql.asotobue-dev.svc.cluster.local - Read/Write connection 


To get the((("passwords", "Helm Charts")))((("Helm", "Charts", "pass- 
words")))((("Charts", "passwords"))) password for "postgres" run: 


export POSTGRES_ADMIN_PASSWORD=$(kubectl get secrets 
--namespace asotobue-dev my-db-postgresql -ob 
jsonpath="{.data.postgresql-postgres-password}" | base64 --decode) 


To get the password for "my-default" run: 


export POSTGRES_PASSWORD=$(kubectl get secret 
--Namespace asotobue-dev my-db-postgresql -ot 
jsonpath="{.data.postgresql-password}" | base64 --decode) 


To connect to your database run the following command: 


kubectl run my-db-postgresql-client --rm --tty -i --restart='Never'& 
--Namespace asotobue-devs 
--image docker.io/bitnami/postgresql:11.14.0-debian-10-r284 
- -env="PGPASSWORD=SPOSTGRES_ PASSWORD" S 
--command -- psql --host my-db-postgresql -U my-default -d mydb 
-p 5432 


To connect to your ((("Helm", "Charts", "connecting to databases")))((("Charts", 
"databases", "connecting to")))((("databases", "connecting to", "Helm 
Charts")))database from outside the cluster execute the following commands: 


kubectl port-forward --namespace asotobue-dev svc/my-db-postgresql 5432:5432 & 
PGPASSWORD="SPOSTGRES_PASSWORD" psql --host 127.0.0.1 -U my-default -d mydb -p 
5432 


Inspect the installation by listing pods, Services, StatefulSets, or Secrets: 
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kubectl get pods 


NAME READY STATUS RESTARTS AGE 
my-db-postgresql-0 1/1 Running 0 23s 


kubectl get services 


NAME TYPE CLUSTER-IP EXTERNAL - IP PORT(S) AGE 
my-db-postgresql ClusterIP 172.30.35.1 <none> 5432/TCP 

3m33s 

my-db-postgresql-headless ClusterIP None <none> 5432/TCP 

3m33s 


kubectl get statefulset 


NAME READY AGE 
my-db-postgresql 1/1 4m24s 


kubectl get secrets 


NAME TYPE DATA AGE 

my-db-postgresql Opaque 2 5m23s 

sh.helm.release.vi.my-db.v1 helm.sh/release.v1 1 5m24s 
Discussion 


When a third party creates a Chart, there is no direct access to default values or the 
list of parameters to override. Helm provides a show command to check these values: 


helm show values bitnami/postgresql 
And shows all the possible values: 


## @section Global parameters 

## Global Docker image parameters 

## Please, note that this will override the image parameters, including dependen 
cies 

## configured to use the global value 

## Current available global Docker image parameters: imageRegistry, imagePullSe 
crets 

## and storageClass 

HH 


## @param global.imageRegistry Global Docker image registry 
## @param global.imagePullSecrets Global Docker registry secret names as an array 
## @param global.storageClass Global StorageClass for Persistent Volume(s) 
## 
global: 
imageRegistry: 
## E.g. 
## imagePullSecrets: 
##  - myRegistryKeySecretName 
HH 
imagePullSecrets: [] 
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5.6 Deploying a Chart with a Dependency 


Problem 
You want to deploy a Helm Chart that is a dependency of another Chart. 


Solution 


Use the dependencies section in the Chart.yaml file to register other Charts. So far, 
we've seen how to deploy simple services to the cluster, but usually a service might 
have other dependencies like a database, mail server, distributed cache, etc. 


In the previous section, we saw how to deploy a PostgreSQL server in a Kubernetes 
cluster. In this section, we'll see how to deploy a service composed of a Java service 
returning a list of songs stored in a PostgreSQL database. The application is summar- 
ized in Figure 5-2. 


Kubernetes Cluster 
PostgreSQL DB 


__ Sng |e 
(json document) service Get songs 


music-db-postgresql 


music music-db-postgresql music-db-postgresql ES 


l music-db-postgresql-headless 


music-67dbf986b7-5xkqm music-db-postgresql-O 


Figure 5-2. Music application overview 


Let’s start creating the Chart layout shown in Recipe 5.1: 


mkdir music 
mkdir music/templates 


cd music 


Then create two template files to deploy the music service. 
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The templates/deployment.yaml file contains the Kubernetes Deployment definition: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
name: {{ .Chart.Name}} 
labels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
{{- if .Chart.AppVersion }} 
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} 
{{- end }} 
spec: 
replicas: {{ .Values.replicaCount }} 
selector: 
matchLabels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
template: 
metadata: 
labels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
spec: 
containers: 
- image: "{{ .Values.image.repository }}:6 
{{ .Values.image.tag | default .Chart.AppVersion}}" 
imagePullPolicy: {{ .Values.image.pullPolicy }} 
name: {{ .Chart.Name}} 
ports: 
- containerPort: {{ .Values.image.containerPort }} 
name: http 
protocol: TCP 
env: 
- name: QUARKUS_DATASOURCE_JDBC_URL 
value: {{ .Values.postgresql.server | & 
default (printf "%s-postgresql" ( .Release.Name )) | quote }} 
- name: QUARKUS_DATASOURCE_USERNAME 
value: {{ .Values.postgresql.postgresqlUsername | & 
default (printf "postgres" ) | quote }} 
name: QUARKUS_DATASOURCE_PASSWORD 
valueFronm: 
secretKeyRef: 
name: {{ .Values.postgresql.secretName | & 
default (printf "%s-postgresql" ( .Release.Name )) | quote }} 
key: {{ .Values.postgresql.secretKey }} 


The templates/service.yaml file contains the Kubernetes Service definition: 


apiVersion: v1 
kind: Service 
metadata: 
labels: 
app.kubernetes.io/name: {{ .Chart.Name }} 
name: {{ .Chart.Name }} 
spec: 
ports: 
- name: http 
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port: {{ .Values.image.containerPort }} 
targetPort: {{ .Values.image.containerPort }} 
selector: 
app.kubernetes.io/name: {{ .Chart.Name }} 


After the creation of the templates, it’s time for the Chart metadata Chart.yaml file. 
In this case, we need to define the dependencies of this Chart too. Since the music 
service uses a PostgreSQL database, we can add the Chart used in Recipe 5.5 as a 
dependency: 

apiVersion: v2 


name: music 
description: A Helm chart for Music service 


type: application 
version: 0.1.0 
appVersion: "1.0.0" 
dependencies: (1) 

- name: postgresql (2) 


version: 10.16.2 © 
repository: "https://charts.bitnami.com/bitnami" (4) 


Dependencies section 
Name of the Chart to add as dependency 


Chart version 


© © 8 Ọ 


Repository 


The final file is Values.yaml with default configuration values. In this case, a new sec- 
tion is added to configure music deployment with PostgreSQL instance parameters: 


image: 
repository: quay.io/gitops-cookbook/music 
tag: "1.0.0" 


pullPolicy: Always 
containerPort: 8080 


replicaCount: 1 


postgresql: (1 
server: jdbc:postgresql://music-db-postgresql:5432/mydb 
postgresqlUsername: my-default 
secretName: music-db-postgresql 
secretKey: postgresql-password 


@ PostgreSQL section 
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With the Chart in place, the next thing to do is download the dependency Chart and 
store it in the charts directory. This process is automatically done by running the 


dependency update command: 
helm dependency update 
The command output shows that one Chart has been downloaded and saved: 


Hang tight while we grab the latest from your chart repositories... 
...Successfully got an update from the "stable" chart repository 
...Successfully got an update from the "bitnami" chart repository 
Update Complete. Happy Helming! # 

Saving 1 charts 

Downloading postgresql from repo https://charts.bitnami.com/bitnami 
Deleting outdated charts 


The directory layout looks like this: 


music 

— Chart. lock 

H Chart. yaml 

| charts 

| L— postgresql-10.16.2.tgz (1) 
ļ— templates 

| ļ— deployment.yaml 

| L— service.yaml 

L— values.yaml 


@ PostgreSQL Chart is placed in the correct directory 


Finally, we deploy the Chart, setting configuration PostgreSQL deployment values 


from the command line: 


helm install music-db --set postgresql.postgresqlPassword=postgres postgresql.post- 


gresqlDatabase=mydb, postgresql.persistence.enabled=false . 
The installation process shows information about the deployment: 


NAME: music-db 

LAST DEPLOYED: Tue Jan 25 17:53:17 2022 
NAMESPACE: default 

STATUS: deployed 

REVISION: 1 

TEST SUITE: None 


Inspect the installation by listing pods, Services, StatefulSets, or Secrets: 


kubectl get pods 


NAME READY STATUS RESTARTS AGE 
music-67dbf986b7-5xkqm 1/1 Running 1 (32s ago) 39s 
music-db-postgresql-0 1/1 Running 0 39s 


kubectl get statefulset 
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NAME READY AGE 
music-db-postgresql 1/1 53s 


kubectl get services 


NAME TYPE CLUSTER-IP EXTERNAL - IP PORT(S) AGE 
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 40d 
music ClusterIP 10.104.110.34 <none> 8080/TCP 82s 
music-db-postgresql ClusterIP 10.110.71.13 <none> 5432/TCP 82s 
music-db-postgresql-headless ClusterIP None <none> 5432/TCP 82s 


We can validate the access to the music service by using port forwarding to the 
Kubernetes Service. 


Open a new terminal window and run the following command: 


kubectl port-forward service/music 8080:8080 


Forwarding from 127.0.0.1:8080 -> 8080 
Forwarding from [::1]:8080 -> 8080 


The terminal is blocked and it’s normal until you stop the kubectl port- forward 
process. Thanks to port forwarding, we can access the music service using the local 
host address and port 8080. 


In another terminal, curl the service: 
curl localhost:8080/song 


The request is sent to the music service deployed in Kubernetes and returns a list of 
songs: 


[ 
{ 
Mud: ay 
"artist": "DT", 
"name": "Quiero Munchies" 
}, 
{ 
"ud": 2, 
"artist": "Lin-Manuel Miranda", 
"name": "We Don't Talk About Bruno" 
}, 
{ 
mud": 3; 
"artist": "Imagination", 
"name": "Just An Illusion" 
}, 
{ 
"id": 4, 
"artist": "Txarango", 
"name": "Tanca Els Ulls" 
}, 
{ 


"id": 5, 
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"artist": "Halsey", 
"name": "Could Have Been Me" 
} 
] 


5.7 Triggering a Rolling Update Automatically 


Problem 


You want to trigger a rolling update of deployment when a ConfigMap object is 
changed. 


Solution 
Use the sha256sum template function to generate a change on the deployment file. 


In Recipe 4.5, we saw that Kustomize has a ConfigMapGenerator that automatically 
appends a hash to the ConfigMap metadata name and modifies the deployment file 
with the new hash when used. Any change on the ConfigMap triggers a rolling update 
of the deployment. 


Helm doesn't provide a direct way like Kustomize does to update a deployment file 
when the ConfigMap changes, but there is a template function to calculate a SHA-256 
hash of any file and embed the result in the template. 


Suppose weve got a Node.js application that returns a greeting message. An environ- 
ment variable configures this greeting message, and in the Kubernetes Deployment, 
this variable is injected from a Kubernetes ConfigMap. 


Figure 5-3 shows an overview of the application. 


Kubernetes Cluster 


Greetings Service 


Aloha Ada —nede_—- QO a Injects 
O deploy 


GREETING = Aloha 


Figure 5-3. Greetings application overview 


Let’s create the Helm Chart for the Greetings application; note that were not covering 
the entire process of creating a Chart, but just the essential parts. You can refer to 
Recipe 5.1 to get started. 
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Create a deployment template that injects a ConfigMap as an environment variable. 
The following listing shows the file: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
name: {{ .Chart.Name}} 
labels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
{{- if .Chart.AppVersion }} 
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} 
{{- end }} 
spec: 
replicas: {{ .Values.replicaCount }} 
selector: 
matchLabels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
template: 
metadata: 
labels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
spec: 
containers: 

- image: "{{ .Values.image.repository }}:& 

{{ .Values.image.tag | default .Chart.AppVersion}}" 
imagePullPolicy: {{ .Values.image.pullPolicy }} 
name: {{ .Chart.Name}} 
ports: 

- containerPort: {{ .Values.image.containerPort }} 
name: http 
protocol: TCP 
env: 
- mame: GREETING 
valueFrom: 
configMapKeyRef : 
name: {{ .Values.configmap.name}} (1) 
key: greeting 


@ ConfigMap name 


@ Property key of the ConfigMap 


The initial ConfigMap file is shown in the following listing: 


apiVersion: v1 
kind: ConfigMap 
metadata: 
name: greeting-config (1) 
data: 
greeting: Aloha (2) 
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@ Sets ConfigMap name 


© Key/value 


Create a Kubernetes Service template to access the service: 


apiVersion: vi 
kind: Service 
metadata: 
labels: 
app.kubernetes.io/name: {{ .Chart.Name }} 
name: {{ .Chart.Name }} 
spec: 
ports: 
- name: http 
port: {{ .Values.image.containerPort }} 
targetPort: {{ .Values.image.containerPort }} 
selector: 
app.kubernetes.io/name: {{ .Chart.Name }} 


Update the values.yaml file with the template configmap parameters: 


image: 
repository: quay.io/gitops-cookbook/greetings 
tag: "1.0.0" 


pullPolicy: Always 
containerPort: 8080 


replicaCount: 1 


configmap: 
Name: greeting-config (1) 


@ Refers to ConfigMap name 
Finally, install the Chart using the install command: 
helm install greetings . 


When the Chart is deployed, use the kubectl port-forward command in one 
terminal to get access to the service: 


kubectl port-forward service/greetings 8080:8080 
And curl the service in another terminal window: 


curl localhost:8080 
Aloha Ada @ 


© Configured greeting is used 
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Now, lets update the ConfigMap file to a new greeting message: 
apiVersion: v1 
kind: ConfigMap 
metadata: 
Name: greeting-config 
data: 
greeting: Hola (13 


@ New greeting message 


Update the appVersion field from the Chart.yaml file to 1.0.1 and upgrade the Chart 
by running the following command: 


helm upgrade greetings . 
Restart the kubectl port-forward process and curl the service again: 


curl localhost:8080 
Aloha Alexandra @ 


@ Greeting message isn’t updated 


The ConfigMap object is updated during the upgrade, but since there are no changes 
in the Deployment object, there is no restart of the pod; hence the environment 
variable is not set to the new value. Listing the pods shows no execution of the rolling 
update: 


kubectl get pods 


NAME READY STATUS RESTARTS AGE 
greetings-64ddfcb649-m5pml 1/1 Running 0 2m21s © 


@ Age value shows no rolling update 


Figure 5-4 summarizes the change. 


Kubernetes Cluster 


Greetings service 


Aloha Ada —nede— CQ a Injects 
Gs) deploy 


GREETING = Hola 


Figure 5-4. Greetings application with new configuration value 


Let’s use the sha256sum function to calculate an SHA-256 value of the configmap.yaml 
file content and set it as a pod annotation, which effectively triggers a rolling update 
as the pod definition has changed: 
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spec: 
replicas: {{ .Values.replicaCount }} 
selector: 
matchLabels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
template: 
metadata: 
labels: 
app.kubernetes.io/name: {{ .Chart.Name}} 
annotations: 
checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml")& 
. | sha256sum }} 1] 


© Includes the configmap.yaml file, calculates the SHA-256 value, and sets it as an 
annotation 


Update the ConfigMap again with a new value: 
apiVersion: vi 
kind: ConfigMap 
metadata: 
Name: greeting-config 
data: 
greeting: Namaste (1) 


@ New greeting message 


Update the appVersion field from Chart.yaml to 1.0.1 and upgrade the Chart by 
running the following command: 


helm upgrade greetings . 
Restart the kubectl port-forward process and curl the service again: 


curl localhost:8080 
Namaste Alexandra @ 


@ Greeting message is the new one 


List the pods deployed in the cluster again, and you'll notice that a rolling update is 
happening: 


kubectl get pods 


NAME READY STATUS RESTARTS AGE 
greetings-5c6b86dbbd-2p9bd 0/1 ContainerCreating 0 3s © 
greetings-64ddfcb649-m5pml 1/1 Running 0 2m21s 


@ A rolling update is happening 
Describe the pod to validate that the annotation with the SHA-256 value is present: 


kubectl describe pod greetings-5c6b86dbbd-s4n7b 
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The output shows all pod parameters. The important one is the annotations placed 
at the top of the output showing the checksum/config annotation containing the 
calculated SHA-256 value: 


Name: greetings - 5c6b86dbbd-s4n7b 

Namespace: asotobue-dev 

Priority: -3 

Priority Class Name: sandbox-users-pods 

Node: ip-10-0-186-34.ec2.internal/10.0.186.34 

Start Time: Thu, 27 Jan 2022 11:55:02 +0100 

Labels: app.kubernetes.io/name=greetings 
pod-tempLate-hash=5c6b86dbbd 

Annotations: checksum/config: 


59e9100616a11d65b691a914cd429dc6011a34e02465173F5f53584b4aa7cbas @ 


@ Calculated value 


Figure 5-5 summarizes the elements that changed when the application was updated. 
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GREETING = Namaste 


Figure 5-5. Final overview of the Greetings application 


5.8 Final Thoughts 


In the previous chapter, we saw Kustomize; in this chapter, weve seen another tool to 
help deploy Kubernetes applications. 


When you need to choose between Kustomize or Helm, you might have questions on 
which one to use. 


In our experience, the best way to proceed is with Kustomize for simple projects, 
where only simple changes might be required between new deployments. 


If the project is complex with external dependencies, and several deployment param- 
eters, then Helm is a better option. 


98 | Chapter 5: Helm 


CHAPTER 6 
Cloud Native CI/CD 


In the previous chapter you learned about Helm, a popular templating system for 
Kubernetes. All the recipes from previous chapters represent a common tooling for 
creating and managing containers for Kubernetes, and now it’s time to think about 
the automation on Kubernetes using such tools. Lets move our focus to the cloud 
native continuous integration/continuous deployment (CI/CD). 


Continuous integration is an automated process that takes new code created by a 
developer and builds, tests, and runs that code. The cloud native CI refers to the 
model where cloud computing and cloud services are involved in this process. The 
benefits from this model are many, such as portable and reproducible workloads 
across clouds for highly scalable and on-demand use cases. And it also represents 
the building blocks for GitOps workflows as it enables automation through actions 
performed via Git. 


Tekton is a popular open source implementation of a cloud native CI/CD system on 
top of Kubernetes. In fact, Tekton installs and runs as an extension on a Kubernetes 
cluster and comprises a set of Kubernetes Custom Resources that define the building 
blocks you can create and reuse for your pipelines.' (See Recipe 6.1.) 


The Tekton engine lives inside a Kubernetes cluster and through its API objects 
represents a declarative way to define the actions to perform. The core components 
such as Tasks and Pipelines can be used to create a pipeline to generate artifacts 
and/or containers from a Git repository (see Recipes 6.2, 6.3, and 6.4). 


Tekton also supports a mechanism for automating the start of a Pipeline with Trig- 
gers. These allow you to detect and extract information from events from a variety of 


1 See the Tekton documentation. 
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sources, such as a webhook, and to start Tasks or Pipelines accordingly (see Recipe 
6.8). 


Working with private Git repositories is a common use case that Tekton supports 
nicely (see Recipe 6.4), and building artifacts and creating containers can be done in 
many ways such as with Buildah (see Recipe 6.5) or Shipwright, which we discussed 
in Chapter 3. It is also possible to integrate Kustomize (see Recipe 6.9) and Helm 
(see Recipe 6.10) in order to make the CI part dynamic and take benefit of the rich 
ecosystem of Kubernetes tools. 


Tekton is Kubernetes-native solution, thus it’s universal; however, it’s not the only 
cloud native CI/CD citizen in the market. Other good examples for GitOps-ready 
workloads are Drone (Recipe 6.11) and GitHub Actions (Recipe 6.12). 


6.1 Install Tekton 


Problem 


You want to install Tekton in order to have cloud native CI/CD on your Kubernetes 
cluster. 


Solution 


Tekton is a Kubernetes-native CI/CD solution that can be installed on top of any 
Kubernetes cluster. The installation brings you a set of Kubernetes Custom Resources 
(CRDs) that you can use to compose your Pipelines, as shown in Figure 6-1: 


Task 
A reusable, loosely coupled number of steps that perform a specific function (e.g., 
building a container image). Tasks get executed as Kubernetes pods, while steps 
in a Task map onto containers. 


Pipeline 
A list Tasks needed to build and/or deploy your apps. 


TaskRun 
The execution and result of running an instance of a Task. 


PipelineRun 
The execution and result of running an instance of a Pipeline, which includes a 
number of TaskRuns. 


Trigger 
Detects an event and connects to other CRDs to specify what happens when such 
an event occurs. 
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Figure 6-1. Tekton Pipelines 
To install Tekton, you just need kubectl CLI and a Kubernetes cluster such as 
Minikube (see Chapter 2). 


Tekton has a modular structure. You can install all components separately or all at 
once (e.g., with an Operator): 


Tekton Pipelines 
Contains Tasks and Pipelines 


Tekton Triggers 
Contains Triggers and EventListeners 


Tekton Dashboard 


A convenient dashboard to visualize Pipelines and logs 


Tekton CLI 


A CLI to manage Tekton objects (start/stop Pipelines and Tasks, check logs) 


6.1 Install Tekton | 101 


You can also use a Kubernetes Operator to install and manage 
Tekton components on your cluster. See more details on how from 
OperatorHub. 


First you need to install the Tekton Pipelines component. At the time of writing this 
book, we are using version 0.37.0: 


kubectl apply \ 
-f https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.37.0/ 
release. yamlL 


The installation will create a new Kubernetes namespace called tekton-pipelines 
and you should see output similar to the following: 


Namespace/tekton-pipelines created 

podsecuritypolicy.policy/tekton-pipelines created 
clusterrole.rbac.authorization.k8s.io/tekton-pipelines-controller-cluster-access 
created 
clusterrole.rbac.authorization.k8s.io/tekton-pipelines-controller -tenant-access 
created 
clusterrole.rbac.authorization.k8s.io/tekton-pipelines-webhook-cluster-access cre- 
ated 

role.rbac.authorization.k8s.io/tekton-pipelines-controller created 
role.rbac.authorization.k8s.io/tekton-pipelines-webhook created 
role.rbac.authorization.k8s.io/tekton-pipelines-leader-election created 
role.rbac.authorization.k8s.io/tekton-pipelines-info created 
serviceaccount/tekton-pipelines-controller created 
serviceaccount/tekton-pipelines-webhook created 
clusterrolebinding.rbac.authorization.k8s.io/tekton-pipelines-controller-cluster- 
access created 
clusterrolebinding.rbac.authorization.k8s.io/tekton-pipelines-controller -tenant- 
access created 
clusterrolebinding.rbac.authorization.k8s.io/tekton-pipelines-webhook-cluster - 
access created 

rolebinding.rbac.authorization.k8s.io0/tekton-pipelines-controller created 
rolebinding. rbac. authorization. k8s.io/tekton-pipelines-webhook created 
rolebinding. rbac. authorization.k8s.io/tekton-pipelines-controller-leaderelection 
created 

rolebinding. rbac. authorization. k8s.io/tekton-pipelines-webhook-leaderelection cre- 
ated 

rolebinding.rbac.authorization.k8s.io/tekton-pipelines-info created 
customresourcedefinition. apiextensions.k8s.io/clustertasks.tekton.dev created 
customresourcedefinition. apiextensions.k8s.io/pipelines.tekton.dev created 
customresourcedefinition.apiextensions.k8s.io/pipelineruns.tekton.dev created 
customresourcedefinition. apiextensions.k8s.io/resolutionrequests.resoLlution.tek- 
ton.dev created 
customresourcedefinition.apiextensions.k8s.io/pipelineresources.tekton.dev created 
customresourcedefinition. apiextensions.k8s.io/runs.tekton.dev created 
customresourcedefinition. apiextensions.k8s.io/tasks.tekton.dev created 
customresourcedefinition.apiextensions.k8s.io/taskruns.tekton.dev created 
secret/webhook-certs created 
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validatingwebhookconfiguration.admissionregistration.k8s.io/validation.web- 
hook.pipeline.tekton.dev created 
mutatingwebhookconfiguration.admissionregistration.k8s.io/webhook.pipeline. tek- 
ton.dev created 
validatingwebhookconfiguration.admissionregistration.k8s.io/config.webhook.pipe- 
line.tekton.dev created 
clusterrole.rbac.authorization.k8s.io/tekton-aggregate-edit created 
clusterrole.rbac.authorization.k8s.io/tekton-aggregate-view created 
configmap/config-artifact-bucket created 

configmap/config-artifact-pvc created 

configmap/config-defaults created 

configmap/feature-flags created 

configmap/pipelines-info created 

configmap/config-leader-election created 

configmap/config-logging created 

configmap/config-observability created 

configmap/config-registry-cert created 
deployment.apps/tekton-pipelines-controller created 
service/tekton-pipelines-controller created 
horizontalpodautoscaler.autoscaling/tekton-pipelines-webhook created 
deployment. apps/tekton-pipelines-webhook created 
service/tekton-pipelines-webhook created 


You can monitor and verify the installation with the following command: 
kubectl get pods -w -n tekton-pipelines 


You should see output like this: 


NAME READY STATUS RESTARTS AGE 
tekton-pipelines-controller-5fd68749f5-tz8dv 1/1 Running 0 3m4s 
tekton- pipelines -webhook - 58dcdbfd9b-hswpk 1/1 Running 0 3m4s 


The preceding command goes in watch mode, thus it remains 
appended. Press Ctrl+C in order to stop it when you see the con- 
troller and webhook pods in Running status. 


Then you can install Tekton Triggers. At the time of writing this book, we are using 
version 0.20.1: 


kubectl apply \ 
-f https://storage.googleapis.com/tekton-releases/triggers/previous/v0.20.1/ 
release. yaml 


You should see the following output: 


podsecuritypolicy.policy/tekton-triggers created 
clusterrole.rbac.authorization.k8s.io/tekton-triggers-admin created 
clusterrole.rbac.authorization.k8s.io/tekton-triggers-core-interceptors created 
clusterrole.rbac.authorization.k8s.io/tekton-triggers-core-interceptors-secrets 
created 

clusterrole.rbac.authorization.k8s.io/tekton-triggers-eventlistener-roles created 
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clusterrole.rbac.authorization.k8s.io/tekton-triggers-eventlistener-clusterroles 
created 

role.rbac.authorization.k8s.io/tekton-triggers-admin created 

role.rbac. authorization. k8s.io/tekton-triggers-admin-webhook created 
role.rbac.authorization.k8s.io/tekton-triggers-core-interceptors created 
role.rbac.authorization.k8s.io/tekton-triggers-info created 
serviceaccount/tekton-triggers-controller created 
serviceaccount/tekton-triggers-webhook created 
serviceaccount/tekton-triggers-core-interceptors created 
clusterrolebinding.rbac.authorization.k8s.io/tekton-triggers-controller-admin cre- 
ated 

clusterrolebinding. rbac.authorization.k8s.io/tekton-triggers-webhook-admin created 
clusterrolebinding.rbac.authorization.k8s.io/tekton-triggers-core-interceptors cre- 
ated 
clusterrolebinding.rbac.authorization.k8s.io/tekton-triggers-core-interceptors- 
secrets created 

rolebinding.rbac. authorization. k8s.io/tekton-triggers-controller-admin created 
rolebinding. rbac. authorization. k8s.io/tekton-triggers-webhook-admin created 
rolebinding.rbac.authorization.k8s.io/tekton-triggers-core-interceptors created 
rolebinding.rbac. authorization. k8s.io/tekton-triggers-info created 
customresourcedefinition. apiextensions.k8s.io/clusterinterceptors.triggers.tek- 
ton.dev created 
customresourcedefinition.apiextensions.k8s.io/clustertriggerbindings. triggers. tek- 
ton.dev created 

customresourcedefinition. apiextensions.k8s.io/eventlisteners.triggers.tekton.dev 
created 

customresourcedefinition. apiextensions.k8s.io/triggers.triggers.tekton.dev created 
customresourcedefinition. apiextensions.k8s.io/triggerbindings.triggers.tekton.dev 
created 

customresourcedefinition. apiextensions.k8s.io/triggertempLates.triggers.tekton.dev 
created 

secret/triggers-webhook-certs created 
validatingwebhookconfiguration.admissionregistration.k8s.io/validation.web- 

hook. triggers.tekton.dev created 

mutatingwebhookconfiguration. admissionregistration.k8s.io/webhook.triggers.tek- 
ton.dev created 
validatingwebhookconfiguration.admissionregistration.k8s.io/config.webhook.trig- 
gers.tekton.dev created 
clusterrole.rbac.authorization.k8s.io/tekton-triggers-aggregate-edit created 
clusterrole.rbac.authorization.k8s.io/tekton-triggers-aggregate-view created 
configmap/config-defaults-triggers created 

configmap/feature-flags-triggers created 

configmap/triggers-info created 

configmap/config-logging-triggers created 

configmap/config-observability-triggers created 

service/tekton-triggers-controller created 
deployment.apps/tekton-triggers-controller created 

service/tekton-triggers-webhook created 

deployment. apps/tekton-triggers-webhook created 

deployment. apps/tekton-triggers-core-interceptors created 
service/tekton-triggers-core-interceptors created 
clusterinterceptor.triggers.tekton.dev/cel created 
clusterinterceptor.triggers.tekton.dev/bitbucket created 
clusterinterceptor.triggers.tekton.dev/github created 
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clusterinterceptor.triggers.tekton.dev/gitlab created 
secret/tekton-triggers-core-interceptors-certs created 


You can monitor and verify the installation with the following command: 
kubectl get pods -w -n tekton-pipelines 


You should see three new pods created and running—tekton-triggers-controller, 
tekton-triggers-core-interceptors, and tekton- triggers -webhook: 


NAME READY STATUS RESTARTS 
AGE 

tekton-pipelines-controller -5fd68749f5-tz8dv 1/1 Running 0 
27m 

tekton-pipelines -webhook - 58dcdbfd9b-hswpk 1/1 Running 0 
27m 

tekton-triggers-controller -854d44fd5d-8jf9q 1/1 Running 0 
105s 

tekton-triggers-core-interceptors-5454f8785f-dhsrb 1/1 Running 0 
104s 

tekton- triggers -webhook-86d75f875-zmjf4 1/1 Running 0 
105s 


After this you have a fully working Tekton installation on top of your Kubernetes 
cluster, supporting Pipelines and automation via event Triggers. In addition to that, 
you could install the Tekton Dashboard in order to visualize Tasks, Pipelines, and 
logs via a nice UI. At the time of writing this book, we are using version 0.28.0: 


kubectl apply \ 
-f https: //storage.googleapis.com/tekton-releases/dashboard/previous/v0.28.0/ 
tekton-dashboard-release. yaml 


You should have output similar to the following: 


customresourcedefinition. apiextensions.k8s.io/extensions.dashboard.tekton.dev cre- 
ated 

serviceaccount/tekton-dashboard created 
role.rbac.authorization.k8s.io/tekton-dashboard-info created 
clusterrole.rbac.authorization.k8s.io/tekton-dashboard-backend created 
clusterrole.rbac.authorization.k8s.io/tekton-dashboard-tenant created 
rolebinding.rbac. authorization. k8s.io0/tekton-dashboard-info created 
clusterrolebinding.rbac.authorization.k8s.io/tekton-dashboard-backend created 
configmap/dashboard-info created 

service/tekton-dashboard created 

deployment.apps/tekton-dashboard created 
clusterrolebinding.rbac.authorization.k8s.io/tekton-dashboard-tenant created 


You can monitor and verify the installation with the following command: 
kubectl get pods -w -n tekton-pipelines 


You should see a new pod created and running—tekton-dashboard: 


NAME READY STATUS RESTARTS 
AGE 
tekton- dashboard -786b6b5579-sscgz 1/1 Running 0 
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2m25s 


tekton-pipelines-controLller -5fd68749f5-tz8dv 1/1 Running 1 (7m16s ago) 
5d7h 

tekton-pipelines -webhook - 58dcdbfd9b-hswpk 1/1 Running 1 (7m6s ago) 
5d7h 

tekton-triggers-controller-854d44fd5d-8jf9q 1/1 Running 2 (7m9s ago) 
Sd7h 

tekton-triggers-core-interceptors-5454f8785f-dhsrb 1/1 Running 1 (7m7s ago) 
Sd7h 

tekton- triggers -webhook -86d75f875-zmjf4 1/1 Running 2 (7m9s ago) 
Sd7h 


By default, the Dashboard is not exposed outside the Kubernetes cluster. You can 
access it by using the following command: 


kubectl port-forward svc/tekton-dashboard 9097:9097 -n tekton-pipelines 
There are several ways to expose internal services in Kubernetes; 


you could also create an Ingress for that as shown in the Tekton 
Dashboard documentation. 


You can now browse to http://localhost:9097 to access your Dashboard, as shown in 
Figure 6-2. 


You can download and install the Tekton CLI for your OS to start creating Tasks 
and Pipelines from the command line. At the time of writing this book, we are using 
version 0.25.0. 


Tekton Dashboard 


OJT. 
5 *. 
@ Tekton resources a « 
About Tekton 
Pipelines 
PipelineRuns Tekton is a powerful and flexible open-source 
framework for creating CI/CD systems, allowing 
developers to build, test, and deploy across cloud 
providers and on-premises systems. 
Environment details 
Dashboard Pipelines Triggers 
Namespace Namespace Namespace 
tekton-pipelines tekton-pipelines tekton-pipelines 
È Import resources Version Version Version 
v0.28.0 v0.37.0 v0.20.1 


© About Tekton 


8 Settings 


Figure 6-2. Tekton Dashboard 


Finally, verify that tkn and Tekton are configured correctly: 


106 | Chapter 6: Cloud Native CI/CD 


tkn version 
You should get the following output: 


Client version: 0.25.0 
Pipeline version: v0.37.0 
Triggers version: v0.20.1 
Dashboard version: v0.28.0 


See Also 


e Tekton Getting Started 


6.2 Create a Hello World Task 


Problem 


You want to start using Tekton by exploring Tasks and creating a sample one. 


Solution 


In Tekton, a Task defines a series of steps that run sequentially to perform logic that 
the Task requires. Every Task runs as a pod on your Kubernetes cluster, with each 
step running in its own container. While steps within a Task are sequential, Tasks can 
be executed inside a Pipeline in parallel. Therefore, Tasks are the building blocks for 
running Pipelines with Tekton. 


Let’s create a Hello World Task: 


apiVersion: tekton.dev/vibeta1 
kind: Task @ 
metadata: 
name: hello @ 
spec: 
steps: © 
- name: say-hello (4) 
image: registry.access.redhat.com/ubi8/ubi (5) 
command : 
- /bin/bash 
args: ['-c', ‘echo Hello GitOps Cookbook reader!'] 


The API as an object of kind Task 
The name of the Task 


The list of steps contained within this Task, in this case just one 


© © 8 8 


The name of the step 
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© The container image where the step starts 

First you need to create this resource in Kubernetes: 
kubectl create -f helloworld-task.yaml 

You should get the following output: 
task. tekton.dev/hello created 


You can verify that the object has been created in your current Kubernetes 
namespace: 


kubectl get tasks 
You should get output similar to the following: 


NAME AGE 
hello 90s 


Now you can start your Tekton Task with tkn CLI: 
tkn task start --showlog hello 
You should get output similar to the following: 


TaskRun started: hello-run-8bmzz 
Waiting for logs to be available... 
[say-hello] Hello World 


A TaskRun is the API representation of a running Task. See Recipe 
6.3 for more details. 


See Also 


e Tekton Task documentation 


6.3 Create a Task to Compile and Package an App from Git 


Problem 


You want to automate compiling and packaging an app from Git on Kubernetes with 
Tekton. 


Solution 


As seen in Recipe 6.2, Tekton Tasks have a flexible mechanism to add a list of 
sequential steps to automate actions. The idea is to create a list of Tasks with a chain 
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of input/output that can be used to compose Pipelines. Therefore a Task can contain a 
series of optional fields for a better control over the resource: 


inputs 
The resources ingested by the Task. 


outputs 
The resources produced by the Task. 


params 
The parameters that will be used in the Task steps. Each parameter has: 


name 
The name of the parameter. 


description 
The description of the parameter. 


default 
The default value of the parameter. 


results 
The names under which Tasks write execution results. 


workspaces 
The paths to volumes needed by the Task. 


volumes 
The Task can also mount external volumes using the volumes attribute. 


The following example, as illustrated in Figure 6-3, shows a Task named build-app 
that clones the sources using the git command and lists the source code in output. 


Params : Outcomes 
— Compile 


Step Step 
Task 


Figure 6-3. build-app Task 


apiVersion: tekton.dev/vibeta1 
kind: Task 
metadata: 

name: build-app 
spec: 

workspaces: (13 

- name: source 
description: The git repo will be cloned onto the volume backing this work 
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space 
params: 
- name: contextDir 
description: the context dir within source 
default: quarkus 
- name: tlsVerify 
description: tls verify 
type: string 
default: "false" 
- name: url 
default: https://github.com/gitops-cookbook/tekton-tutorial-greeter.git 
- name: revision 
default: master 
name: subdirectory 
default: "" 
- name: sslVerify 
description: defines if http.sslVerify should be set to true or false in the 
global git config 
type: string 
default: "false" 
steps: 
- image: 'gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git- 
init:v0.21.0' 
name: clone 
resources: {} 
script: | 
CHECKOUT_DIR="$(workspaces.source.path)/$(params.subdirectory)" 
cleandir() { 
# Delete any existing contents of the repo directory if it exists. 
# 
# We don't just "rm -rf SCHECKOUT_DIR" because $CHECKOUT_DIR might be "/" 
# or the root of a mounted volume. 
if [[ -d "SCHECKOUT_DIR" ]] ; then 
# Delete non-hidden files and directories 
rm -rf "SCHECKOUT_DIR"/* 
# Delete files and directories starting with . but excluding .. 
rm -rf "$CHECKOUT_DIR"/.[!.]* 
# Delete files and directories starting with .. plus any other charac 


ter 
rm -rf "$CHECKOUT_DIR"/..2* 
fi 
} 
/ko-app/git-init \ 
-url "S(params.url)" \ 
-revision "$(params.revision)" \ 
-path "S$CHECKOUT_DIR" \ 
-sslVerify="$(params.sslVerify)" 
cd "SCHECKOUT_DIR" 
RESULT_SHA="$(git rev-parse HEAD)" 
- name: build-sources 
image: gcr.io/cloud-builders/mvn 
command: 
- mvn 
args: 
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- -DskipTests 
- clean 
- install 
env: 
- name: user.home 
value: /home/tekton 
workingDir: "/workspace/source/$(params.contextDir)" 


© A Task step and Pipeline Task can share a common filesystem via a Tekton 
workspace. The workspace could be either backed by something like Persistent- 
VolumeClaim (PVC) and a ConfigMap, or just ephemeral (emptyDir). 


© A Task can have parameters; this feature makes the execution dynamic. 
Let’s create the Task with the following command: 

kubectl create -f build-app-task.yaml 
You should get output similar to the following: 

task. tekton.dev/build-app created 


You can verify that the object has been created in your current Kubernetes 
namespace: 


kubectl get tasks 
You should get output similar to the following: 


NAME AGE 
build-app 3s 


You can also list the Task with the tkn CLI: 
tkn task ls 
You should get output similar to the following: 


NAME DESCRIPTION AGE 

build-app 10 seconds ago 
When you start a Task, a new TaskRun object is created. TaskRuns are the API 
representation of a running Task; thus you can create it with the tkn CLI using the 
following command: 


tkn task start build-app \ 
--param contextDir='quarkus' \ 
--workspace name=source,emptyDir=""_ \ 
--showLog 
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When parameters are used inside a Task or Pipeline, you will be 
prompted to add new values or confirm default ones, if any. In 
order to use the default values from the Task defintion without 
prompting for values, you can use the --use-param-defaults 
option. 


You should get output similar to the following: 


? Value for param `tlsVerify` of type ‘string’? (Default is `false`) false 

? Value for param ‘url’ of type ‘string’? (Default is ‘https:// 
github.com/gitops-cookbook/tekton-tutorial-greeter.git’) https://github.com/gitops- 
cookbook/tekton-tutorial-greeter.git 

? Value for param ‘revision’ of type ‘string’? (Default is `master`) master 

? Value for param ‘subdirectory’ of type ‘string’? (Default is ``) 

? Value for param ‘sslVerify’ of type ‘string’? (Default is `false`) false 

TaskRun started: build-app-run-rzcd8 

Waiting for logs to be available... 

[clone] {"Level":"info","ts":1659278019.0018234,"caller":"git/ 
git.go:169","msg":"Successfully cloned https://github.com/gitops-cookbook/tekton- 
tutorial-greeter.git @ d9291c456db1ce29177b77ffeaa9b71ad80a50e6 (grafted, HEAD, ori 
gin/master) in path /workspace/source/"} 

[clone] {"lLevel":"info","ts":1659278019.0227938,"caller":"git/ 
git.go:207","msg":"Successfully initialized and updated submodules in path /work 
space/source/"} 


[build-sources] [INFO] Scanning for projects... 

[build-sources] Downloading from central: https://repo.maven.apache.org/maven2/io/ 
quarkus/quarkus-universe-bom/1.6.1.Final/quarkus-universe-bom-1.6.1.Final.pom 
Downloaded from central: https://repo.maven.apache.org/maven2/io/quarkus/quarkus- 
universe-bom/1.6.1.Final/quarkus-universe-bom-1.6.1.Final.pom (412 kB at 118 kB/s) 
[build-sources] [INFO] 


[build-sources] [INFO] Installing /workspace/source/quarkus/target/tekton-quarkus- 
greeter.jar to /root/.m2/repository/com/redhat/developers/tekton-quarkus-greeter/ 
1.0.0-SNAPSHOT/tekton-quarkus-greeter-1.0.0-SNAPSHOT. jar 

[build-sources] [INFO] Installing /workspace/source/quar - 

kus/pom.xml to /root/.m2/repository/com/redhat/developers/tekton-quarkus-greeter/ 
1.0.0-SNAPSHOT/tekton-quarkus-greeter-1.0.0-SNAPSHOT. pom 

[build-sources] [INFO] 

[build-sources] [INFO] BUILD SUCCESS 

[build-sources] [INFO] 

[build-sources] [INFO] Total time: 04:41 min 

[build-sources] [INFO] Finished at: 2022-07-31T14:38:22Z 

[build-sources] [INFO] 


Or, you can create a TaskRun object manually like this: 


apiVersion: tekton.dev/vibeta1 
kind: TaskRun 
metadata: 
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generateName: build-app-run- (13 
labels: 
app.kubernetes.io/managed-by: tekton-pipelines 
tekton.dev/task: build-app 
spec: 
params: 
- name: contextDir 
value: quarkus 
- name: revision 
value: master 
- name: sslVerify 
value: "false" 
- name: subdirectory 
value: "" 
- name: tlsVerify 
value: "false" 
- name: url 
value: https://github.com/gitops-cookbook/tekton-tutorial-greeter.git 
taskRef: @ 
kind: Task 
name: build-app 
workspaces: 
- emptyDir: {} 
name: source 


@ If you don't want to specify a name for each TaskRun, you can use the generate 
Name attribute to let Tekton pick a random one from the string you defined. 


© Here you list the Task that the TaskRun is referring to. 
And start it in this way: 
kubectl create -f build-app-taskrun.yaml 
You should get output similar to the following: 
taskrun.tekton.dev/build-app-run-65vmh created 
You can also verify it with the tkn CLI: 


tkn taskrun ls 


You should get output similar to the following: 


NAME STARTED DURATION STATUS 
build-app-run-65vmh 1 minutes ago 2m37s Succeeded 
build-app-run-rzcd8 2 minutes ago 3m58s Succeeded 


You can get the logs from the TaskRun by specifying the name of the TaskRun: 


tkn taskrun logs build-app-run-65vmh -f 
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See Also 
Debugging a TaskRun 


6.4 Create a Task to Compile and Package an App from 
Private Git 


Problem 


You want to use a private Git repository to automate compiling and packaging of an 
app on Kubernetes with Tekton. 


Solution 


In Recipe 6.3 you saw how to compile and package a sample Java application using 
a public Git repository, but most of the time people deal with private repos at work, 
so how do you integrate them? Tekton supports the following authentication schemes 
for use with Git: 


e Basic-auth 
e SSH 


With both options you can use a Kubernetes Secret to store your credentials and 
attach them to the ServiceAccount running your Tekton Tasks or Pipelines. 


Tekton uses a default service account, however you can override it 
following the documentation here. 


Let’s start with a common example of basic authentication and a popular Git service 
such as GitHub. 


GitHub uses personal access tokens (PATs) as an alternative to 
using passwords for authentication. You can use a PAT instead of a 
clear-text password to enhance security. 
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First you need to create a Secret. You can do this by creating the following YAML file: 


apiVersion: v1 
kind: Secret 
metadata: 

Name: github-secret 

annotations: 

tekton.dev/git-0: https://github.com (13 

type: kubernetes.io/basic-auth (2) 
stringData: 

username: YOUR_USERNAME © 

password: YOUR PASSWORD @ 


@ Here you specify the URL for which Tekton will use this Secret, in this case 
GitHub 


© This is the type of Secret, in this case a basic authentication one 
© Your Git user, in this case your GitHub user 


@ You Git password, in this case your GitHub personal access token 
You can now create the Secret with the following command: 
kubectl create -f git-secret.yaml 
You should get the following output: 
secret/git-secret created 
You can also avoid writing YAML and do everything with kubect1 as follows: 


kubectl create secret generic git-secret \ 
--type=kubernetes.io/basic-auth \ 
--from-literal=username=YOUR_USERNAME \ 
--from-literal=password=YOUR_PASSWORD 


And then you just annotate the Secret as follows: 
kubectl annotate secret git-secret "tekton.dev/git-0=https://github.com" 
Once you have created and annotated your Secret, you have to attach it to the 
ServiceAccount running your Tekton Tasks or Pipelines. 
Let’s create a new ServiceAccount for this purpose: 


apiVersion: vi 
kind: ServiceAccount 
metadata: 

name: tekton-bot-sa 
secrets: 

- name: git-secret (1) 


@ List of Secrets attached to this ServiceAccount 


6.4 Create a Task to Compile and Package an App from Private Git | 115 


kubectl create -f tekton-bot-sa.yaml 


You should get the following output: 


serviceaccount/tekton-bot-sa created 


You can create the ServiceAccount directly with kubectl as 
follows: 


kubectl create serviceaccount tekton-bot-sa 
and then patch it to add the secret reference: 


kubectl patch serviceaccount tekton-bot-sa -p 
'{"secrets": [{"name": "git-secret"}]}' 


Once credentials are set up and linked to the ServiceAccount running Tasks or Pipe- 
lines, you can just add the --serviceaccount=<NAME> option to your tkn command, 
using the Recipe 6.3 example: 


tkn task start build-app \ 
--serviceaccount='tekton-bot-sa' \ (1) 
--param url='https://github.com/gitops-cookbook/tekton-greeter-private.git' \ (2) 
--param contextDir='quarkus' \ 
--workspace name=source,emptyDir=""_\ 
--showLlog 


Here you specify the ServiceAccount to use; this will override the default one at 
runtime. 


Here you can override the default repository with one of your choice. In this 
example there’s a private repository that you cannot access, but you can create a 
private repository on your own and test it like this. 


You should get output similar to the following: 


[clone] {"level":"info","ts":1659354692.1365478,"caller":"git/ 
git.go:169","msg":"Successfully cloned https://github.com/gitops-cookbook/tekton- 
greeter-private.git @ 5250e1fa185805373e620d1c04a0c48129efd2ee (grafted, HEAD, ori 
gin/master) in path /workspace/source/"} 

[clone] {"Level":"info","ts":1659354692.1546066,"caller":"git/ 
git.go:207","msg":"Successfully initialized and updated submodules in path /work 
space/source/"} 


[build-sources] [INFO] 

[build-sources] [INFO] BUILD SUCCESS 

[build-sources] [INFO] 

[build-sources] [INFO] Total time: 04:30 min 
[build-sources] [INFO] Finished at: 2022-07-31T15:30:01Z 
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[build-sources] [INFO] 


See Also 


e Tekton Authentication 


6.5 Containerize an Application Using a Tekton Task 
and Buildah 


Problem 


You want to compile, package, and containerize your app with a Tekton Task on 
Kubernetes. 


Solution 


Automation is essential when adopting the cloud native approach, and if you decide 
to use Kubernetes for your CI/CD workloads, you need to provide a way to package 
and deploy your applications. 


In fact, Kubernetes per se doesnt have a built-in mechanism to build containers; it 
just relies on add-ons such as Tekton or external services for this purpose. That’s 
why in Chapter 3 we did an overview on how to create containers for packaging 
applications with various open source tools. In Recipe 3.3 we used Buildah to create a 
container from a Dockerfile. 


Thanks to Tekton’s extensible model, you can reuse the same Task defined in Recipe 
6.3 to add a step to create a container using the outcomes from the previous steps, as 
shown in Figure 6-4. 


Params y Build Push Results 
« = (JAR) (Container Image) 


Step Step 
Task 


Figure 6-4. Build Push app 


The container can be pushed to a public container registry such as DockerHub or 
Quay.io, or to a private container registry. Similar to what we have seen in Recipe 6.4 
for private Git repositories, pushing a container image to a container registry needs 
authentication. A Secret needs to be attached to the ServiceAccount running the 
Task as follows. See Chapter 2 for how to register and use a public registry. 
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kubectl create secret docker-registry container-registry-secret \ 
--docker-server='YOUR_REGISTRY_SERVER' \ 
--docker-username='YOUR_REGISTRY_USER' \ 
- -docker -password='YOUR_REGISTRY_PASS' 


secret/container-registry-secret created 


Verify it is present and check that the Secret is of type kubernetes.io/dockercon 
figjson: 


kubectl get secrets 
You should get the following output: 


NAME TYPE DATA AGE 
container-registry-secret kubernetes.io/dockerconfigjson 1 1s 


Let’s create a ServiceAccount for this Task: 
kubectl create serviceaccount tekton-registry-sa 
Then let’s add the previously generated Secret to this ServiceAccount: 


kubectl patch serviceaccount tekton-registry-sa \ 
-p '{"secrets": [{"name": "container-registry-secret"}]}' 


You should get the following output: 
serviceaccount/tekton-registry-sa patched 


Let’s add a new step to create a container image and push it to a container registry. 
In the following example we use the book’s organization repos at Quay.io—quay.io/ 
gitops-cookbook/tekton-greeter: Latest: 


apiVersion: tekton.dev/vibeta1 
kind: Task 
metadata: 
name: build-push-app 
spec: 
workspaces: 
- Mame: source 
description: The git repo will be cloned onto the volume backing this work 
space 
params: 
- name: contextDir 
description: the context dir within source 
default: quarkus 
- name: tlsVerify 
description: tls verify 
type: string 
default: "false" 
- name: url 
default: https://github.com/gitops-cookbook/tekton-tutorial-greeter.git 
- name: revision 
default: master 
name: subdirectory 
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default: "" 
- name: sslVerify 
description: defines if http.sslVerify should be set to true or false in the 


global git config 


type: string 
default: "false" 
- name: storageDriver 
type: string 
description: Storage driver 
default: vfs 
- name: destinationImage 
description: the fully qualified image name 
default: "" 


steps: 


- image: 'gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git- 


LNLE VO. 21.0" 


ter 


name: clone 
resources: {} 
script: | 
CHECKOUT_DIR="$(workspaces.source.path)/$(params.subdirectory)" 
cleandir() { 
# Delete any existing contents of the repo directory if it exists. 
# 
# We don't just "rm -rf SCHECKOUT_DIR" because $CHECKOUT_DIR might be "/" 
# or the root of a mounted volume. 
if [[ -d "SCHECKOUT_DIR" ]] ; then 
# Delete non-hidden files and directories 
rm -rf "SCHECKOUT_DIR"/* 
# Delete files and directories starting with . but excluding .. 
rm -rf "SCHECKOUT_DIR"/.[!.]* 
# Delete files and directories starting with .. plus any other charac 


rm -rf "SCHECKOUT_DIR"/..?* 
fi 
} 
/ko-app/git-init \ 
-url "S(params.url)" \ 
-revision "$(params.revision)" \ 
-path "$CHECKOUT_DIR" \ 
-sslVerify="$(params.sslVerify)" 
cd "S$CHECKOUT_DIR" 
RESULT_SHA="$(git rev-parse HEAD)" 
- name: build-sources 
image: gcr.io/cloud-builders/mvn 
command: 
- mvn 
args: 
- -DskipTests 
- clean 
- install 
env: 
- name: user.home 
value: /home/tekton 
workingDir: "/workspace/source/$(params.contextDir)" 
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- name: build-and-push-image 
image: quay.io/buildah/stable 
script: | 
#!/usr/bin/env bash 
buildah --storage-driver=$STORAGE_DRIVER --tls-verify=$(params.tlsVerify) 
bud --layers -t $DESTINATION_IMAGE $CONTEXT_DIR 
buildah --storage-driver=$STORAGE_DRIVER --tls-verify=$(params.tlsVerify) 
push SDESTINATION_IMAGE docker: //SDESTINATION_IMAGE 
env: 
- mame: DESTINATION_IMAGE 
value: "S(params.destinationImage)" 
- mame: CONTEXT_DIR 
value: "/workspace/source/$(params.contextDir)" 
- mame: STORAGE_DRIVER 
value: "$(params.storageDriver)" 
workingDir: "/workspace/source/$(params.contextDir)" 
volumeMounts: 
- name: varlibc 
mountPath: /var/lib/containers 
volumes: 
- name: varlibc 
emptyDir: {} 


Leťs create this Task: 
kubectl create -f build-push-app.yaml 
You should get the following output: 
task.tekton.dev/build-push-app created 


Now let’s start the Task with the Buildah step creating a container image and with a 
new parameter destinationImage to specify where to push the resulting container 
image: 


tkn task start build-push-app \ 
--serviceaccount='tekton-registry-sa' \ 
--param url='https://github.com/gitops-cookbook/tekton-tutorial-greeter.git' \ 
--param destinationImage='quay.io/gitops-cookbook/tekton-greeter:latest' \ (1) 
--param contextDir='quarkus' \ 
--workspace name=source,emptyDir="" \ 
--use-param-defaults \ 
--showLlog 


@ Here you can place your registry; in this example we are using the book’s organi- 
zation repos at Quay.io. 


You should get output similar to the following: 


Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/ 
plexus-utils/3.0.5/plexus-utils-3.0.5.jar (230 kB at 301 kB/s) 

[build-sources] [INFO] Installing /workspace/source/quarkus/target/tekton-quarkus- 
greeter.jar to /root/.m2/repository/com/redhat/developers/tekton-quarkus-greeter/ 
1.0.0-SNAPSHOT/tekton-quarkus-greeter-1.0.0-SNAPSHOT. jar 
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[build-sources] [INFO] Installing /workspace/source/quar- 


1.0.0-SNAPSHOT/tekton-quarkus-greeter-1.0.0-SNAPSHOT.pom 
[build-sources] [INFO] 

[build-sources] [INFO] BUILD SUCCESS 

[build-sources] [INFO] 

[build-sources] [INFO] Total time: 02:59 min 
[build-sources] [INFO] Finished at: 2022-08-02T06:18:37Z 
[build-sources] [INFO] 


[build-and-push-image] Trying to pull registry.access.redhat.com/ubi8/ 
openjdk-11:latest... 

[build-and-push-image] Getting image source signatures 
[build-and-push-image] Checking if image destination supports signatures 
[build-and-push-image] Copying blob 

sha256: 1e09a5ee0038fbe06a18e7F355188bbabc387467144abcd435f7544fef395aal 
[build-and-push-image] Copying blob 
sha256:0d725b91398ed3db11249808d89e688e62e511bbd4a2e875ed8493ce1febdb2c 
[build-and-push-image] Copying blob 

sha256: 1e09a5ee0038fbe06a18e7F355188bbabc387467144abcd435f7544fef395aal1 
[build-and-push-image] Copying blob 
sha256:0d725b91398ed3db11249808d89e688e62e511bbd4a2e875ed8493celfebdb2c 
[build-and-push-image] Copying blob 
sha256:e441d34134fac91baa79be3e2bb8fb3dba71ba5c1ea012cb5daeb7180a054687 
[build-and-push-image] Copying blob 
sha256:e441d34134fac91baa79be3e2bb8fb3dba71ba5c1ea012cb5daeb7180a054687 
[build-and-push-image] Copying config 
sha256:0c308464b19eaa9a01c3fdd6b63a043c160d4eea85e461bbbb7d01d168f6d993 
[build-and-push-image] Writing manifest to image destination 
[build-and-push-image] Storing signatures 

[build-and-push-image] STEP 2/2: COPY target/quarkus-app /deployments/ 


[build-and-push-image] --> 42fe38b4346 


greeter: Latest 

[build-and-push-image ] 
42fe38b43468c3ca32262dbea6fd78919aba2bd35981cd4f71391e07786c9e21 
[build-and-push-image] Getting image source signatures 
[build-and-push-image] Copying blob 
sha256:647a854c512bad44709221b6b0973e884f 29bcb5a380ee32e95bfb0189b620e6 
[build-and-push-image] Copying blob 

sha256: f2ee6b2834726167d0de06f 3bbe65962aef 79855c5ede0d2ba93b4408558d9c9 
[build-and-push-image] Copying blob 
sha256:8e0e04b5c700a86f4a112F41e7e767a9d7c539Fe3391611313bf 76edb07eeab1 
[build-and-push-image] Copying blob 
sha256:69c55192bed92cbb669c88eb3c36449b64ac93ae466abfff2a575273ce05a39e 
[build-and-push-image] Copying config 
sha256:42fe38b43468c3ca32262dbea6fd78919aba2bd35981cd4f71391e07786c9e21 
[build-and-push-image] Writing manifest to image destination 
[build-and-push-image] Storing signatures 


[build-and-push-image] COMMIT quay.io/gitops-cookbook/tekton-greeter: latest 


[build-and-push-image] Successfully tagged quay.io/gitops-cookbook/tekton- 


[build-and-push-image] STEP 1/2: FROM registry.access.redhat.com/ubi8/openjdk-11 


kus/pom.xml to /root/.m2/repository/com/redhat/developers/tekton-quarkus-greeter/ 
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See Also 


e Buildah 


e Docker Authentication for Tekton 


6.6 Deploy an Application to Kubernetes Using a 
Tekton Task 


Problem 


You want to deploy an application from a container image to Kubernetes with a 
Tekton Task. 


Solution 


While in Recipes 6.3, 6.4, and 6.5 we have listed a Tekton Task that is useful for 
continuous integration (CI), in this recipe we'll start having a look at the Continous 
Deployment (CD) part by deploying an existing container image to Kubernetes. 


We can reuse the container image we created and pushed in Recipe 6.5, available at 
quay.io/gitops-cookbook/tekton-greeter: latest: 


apiVersion: tekton.dev/vibeta1 
kind: Task 
metadata: 
name: kubectl 
spec: 
params: 
- mame: SCRIPT 
description: The kubectl CLI arguments to run 
type: string 
default: "kubectl help" 
steps: 
- Mame: oc 
image: quay.io/openshift/origin-cli: latest 1) 
script: | 
#!/usr/bin/env bash 


$(params. SCRIPT) 
@ For this example we are using kubectl from this container image, which also 


contains OpenShift CLI and it has an smaller size compared to gcr.io/cloud- 
builders/kubectl. 


Let’s create this Task: 


kubectl create -f kubectl-task.yaml 
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You should get the following output: 
task. tekton.dev/kubectl created 


As discussed in Recipe 6.5, Tekton uses a default ServiceAccount for running Tasks 
and Pipelines, unless a specific one is defined at runtime or overridden at a global 
scope. The best practice is always to create a specific ServiceAccount for a particular 
action, so let’s create one named tekton-deplLoyer -sa for this use case as follows: 


kubectl create serviceaccount tekton-deployer-sa 
You should get the following output: 
serviceaccount/tekton-deployer-sa created 


A ServiceAccount needs permission to deploy an application to Kubernetes. Roles 
and RoleBindings are API objects used to map a certain permission to a user or a 
ServiceAccount. 


You first define a Role named pipeline-role for the ServiceAccount running the 
Tekton Task with permissions to deploy apps: 


apiVersion: rbac.authorization.k8s.io/v1 
kind: Role 
metadata: 
name: task-role 
rules: 
- apiGroups: 
resources: 
- pods 
- services 
- endpoints 
- configmaps 
- secrets 
verbs: 
a Spn 
- apiGroups: 
- apps 
resources: 
- deployments 
- replicasets 
verbs: 
= Men 
- apiGroups: 
resources: 
- pods 
verbs: 
- get 
- apiGroups: 
- apps 
resources: 
- replicasets 
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verbs: 
- get 


Now you need to bind the Role to the ServiceAccount: 


apiVersion: rbac.authorization.k8s.io/v1 
kind: RoleBinding 
metadata: 

name: task-role-binding 
roleRef: 

kind: Role 

name: task-role 

apiGroup: rbac.authorization.k8s.io 
subjects: 

- kind: ServiceAccount 

name: tekton-deployer-sa 


Now you can create the two resources as follows: 


kubectl create -f task-role.yaml 
kubectl create -f task-role-binding.yaml 


You should get the following output: 


role.rbac.authorization.k8s.io/task-role created 
rolebinding.rbac.authorization.k8s.io0/task-role-binding created 


Finally, you can define a TaskRun as follows: 


apiVersion: tekton.dev/vibeta1 
kind: TaskRun 
metadata: 
name: kubectl-taskrun 
spec: 
serviceAccountName: tekton-depLloyer-sa 
taskRef: 
name: kubectl 
params: 
- mame: SCRIPT 
value: | 
kubectl create deploy tekton-greeter --image=quay.io/gitops-cookbook/ 
tekton-greeter: latest 


And run it in this way: 
kubectl create -f kubectl-taskrun.yaml 
You should get the following output: 
taskrun.tekton.dev/kubectl-run created 
You can check the logs to see the results: 
tkn taskrun logs kubectl-run -f 


You should get output similar to the following: 
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? Select taskrun: kubectl-run started 9 seconds ago 
[oc] deployment.apps/tekton-greeter created 


After a few seconds you should see the Deployment in Ready state: 


kubectl get deploy 


NAME READY UP-TO-DATE AVAILABLE AGE 
tekton-greeter 1/1 1 0 30s 


The first time might take a while due to the time it takes to pull the 
container image. 


Check if the app is available, expose the Deployment, and forward Kubernetes traffic 
to your workstation to test it: 


kubectl expose deploy/tekton-greeter --port 8080 
kubectl port-forward svc/tekton-greeter 8080:8080 


In another terminal, run this command: 
curl localhost: 8080 
You should see the following output: 


Meeow!! from Tekton ---- 


See Also 


e Tekton Task 


6.7 Create a Tekton Pipeline to Build and Deploy an App 
to Kubernetes 


Problem 


You want to create a Pipeline to compile, package, and deploy an app on Kubernetes 
with Tekton. 


Solution 


In the previous recipes we have seen how to create Tasks to execute one or more steps 
sequentially to build apps. In this recipe we will discuss Tekton Pipelines, a collection 
of Tasks that you can define and compose in a specific order of execution, either 
sequentially or in parallel, as you can see in Figure 6-5. 
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Pipeline 


Figure 6-5. Tekton Pipelines flows 


Tekton Pipelines supports parameters and a mechanism to exchange outcomes 
between different Tasks. For instance, using the examples shown in Recipes 6.5 and 
6.6: 


kubectl patch serviceaccount tekton-deployer-sa \ 
-p '{"secrets": [{"name": "container-registry-secret"}]}' 


apiVersion: tekton.dev/vibeta1 
kind: Pipeline 
metadata: 
name: tekton-greeter-pipeline 
spec: 
params: 1] 
- mame: GIT_REPO 
type: string 
- mame: GIT_REF 
type: string 
- mame : DESTINATION_IMAGE 
type: string 
- mame : SCRIPT 
type: string 
tasks: @ 
- name: build-push-app 
taskRef: © 
name: build-push-app 
params: 
- mame: url 
value: "S(params.GIT_REPO)" 
- name: revision 
value: "S(params.GIT_REF)" 
- name: destinationImage 
value: "S(params.DESTINATION_IMAGE)" 
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workspaces: 

- Mame: source 
name: deploy-app 
taskRef: 

name: kubectl 
params: 

- name: SCRIPT 

value: "S(params.SCRIPT)" 
workspaces: 

- Mame: source 
runAfter: @ 

- build-push-app 
workspaces: 

- Mame: source 


Pipeline parameters 
A list of Tasks for the Pipeline 


The exact name of the Task to use 


© © 8 8 


You can decide the order with the runAfter field to indicate that a Task must 
execute after one or more other Tasks 


© One or more common Workspaces used to share data between Tasks 
Let's create the Pipeline as follows: 

kubectl create -f tekton-greeter-pipeline. yaml 
You should get the following output: 

pipeline. tekton.dev/tekton-greeter-pipeline created 


Similarly to TaskRuns, you can run this Pipeline by creating a PipelineRun resource 
as follows: 


apiVersion: tekton.dev/vibeta1 
kind: PipelineRun 
metadata: 
generateName: tekton-greeter-pipeline-run- 
spec: 
params: 
- mame: GIT_REPO 
value: https://github.com/gitops-cookbook/tekton-tutorial-greeter.git 
- name: GIT_REF 
value: "master" 
- name: DESTINATION_IMAGE 
value: "quay.io/gitops-cookbook/tekton-greeter: latest" 
- name: SCRIPT 
value: | 
kubectl create deploy tekton-greeter --image=quay.io/gitops-cookbook/ 
tekton-greeter: latest 
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pipelineRef: 
name: tekton-greeter-pipeline 
workspaces: 
- mame: source 
emptyDir: {} 


You can run the Pipeline by creating this PipelineRun object as follows: 
kubectl create -f tekton-greeter-pipelinerun. yaml 
You can check the status: 


tkn pipelinerun ls 


NAME STARTED DURATION STATUS 
tekton-greeter-pipeline-run-ntl5r 7 seconds ago --- Running 
Now that you have seen how to reuse existing Tasks within a Pipeline, its a good time 
to introduce the Tekton Hub, a web-based platform for developers to discover, share, 
and contribute Tasks and Pipelines for Tekton (see Figure 6-6). 


Welcome to Tekton Hub 


Discover, search and share reusable Tasks and Pipelines 


G *s em we om *5 cm es em *s 


Buildpacks vo2 git cli voa golang build vo3 jenkins operation vor pull request vor 


The Buildpacks pipeline builds 


This Task is Golang task to build The following task can be used to This Tash 
Go projects. interact with the the Jenkins REST 
APL 


Bulldpaeks. 
Updated a month ago Updated 6 days ago Updated 8 months ago Updated 9 months ago 

=D © cD epDanD 

om nái om ws om we om wa om ws 
RedHat CodeReady ssh remote commands vo) rsync vol send-to-telegram vol trigger jenkins job vo 


vol 
Dependency Analysis The following task can be used to This task can be used to These ta: 


synchronize local and remote files 


simple ‘The following task can be used to 
telegram chat. This trigger a 
Bot API of telegram sl 
sage. 


D Developer Toots The RedHat CodeReady 
O Git 


D Image Build 
O Integration & Delivery 


Figure 6-6. Tekton Hub 


You can implement the same Pipeline with Tasks already available in the Hub. In our 
case, we have: 


git-clone 
Task that clones a repo from the provided URL into the output Workspace. 


buildah 
Task that builds source into a container image and can push it to a container 
registry. 
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kubernetes-actions 
The generic kubectl CLI task, which can be used to run all kinds of k8s 
commands. 


First let’s add them to our namespace as follows: 


tkn hub install task git-clone 

tkn hub install task maven 

tkn hub install task buildah 

tkn hub install task kubernetes-actions 


You should get output similar to the following to confirm they are installed properly 
in your namespace: 


Task git-clone(0.7) installed in default namespace 

Task maven(0.2) installed in default namespace 

Task buildah(0.4) installed in default namespace 

Task kubernetes-actions(0.2) installed in default namespace 


You can cross-check it with the following command: 
kubectl get tasks 


You should get output similar to the following: 


NAME AGE 
buildah 50s 
git-clone 52s 
kubernetes-actions 49s 
maven 51s 


Some Tekton installations like the one made with the Operator for 
OpenShift Pipelines provide a common list of useful Tasks such as 
those just listed, provided as ClusterTasks. ClusterIasks are Tasks 
available for all namespaces within the Kubernetes cluster. Check 
if your installation already provides some with this command: 
kubectl get clustertasks. 


Now the Pipeline has four Tasks, as you can see in Figure 6-7. 


Params Results 
—>| A fetch-repo build-app build-push-image deploy 


Task Task Task Task 


Pipeline 


Figure 6-7. Pipeline 
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In this example you'll see a PersistentVolumeClaim as a Workspace because here the 
data is shared among different Tasks so we need to persist it: 


apiVersion: v1 
kind: PersistentVolumeClaim 
metadata: 
Name: app-source-pvc 
spec: 
accessModes: 
- ReadWriteOnce 
resources: 
requests: 
storage: 1Gi 


As usual, you can create the resource with kubectl: 
kubectl create -f app-source-pvc.yaml 

You should see the following output: 
persistentvolumeclaim/app-source-pvc created 


kubectl get pvc 


NAME STATUS VOLUME CAPACITY 
ACCESS MODES STORAGECLASS AGE 

app-source-pvc Bound pvc-e85ade46-aaca-4f3f-b644-d8ff99Fd9d5e =1Gi 

RWO standard 61s 


In Minikube you have a default StorageClass that provides dynamic 
storage for the cluster. If you are using another Kubernetes cluster, 
please make sure you have a dynamic storage support. 


The Pipeline definition now is: 


apiVersion: tekton.dev/vibeta1 
kind: Pipeline 
metadata: 
name: tekton-greeter-pipeline-hub 
spec: 
params: 
- default: https://github.com/gitops-cookbook/tekton-tutorial-greeter.git 
name: GIT_REPO 
type: string 
- default: master 
name: GIT_REF 
type: string 
- default: quay.io/gitops-cookbook/tekton-greeter: latest 
name: DESTINATION_IMAGE 
type: string 
- default: kubectl create deploy tekton-greeter --image=quay.io/gitops-cookbook/ 
tekton-greeter: latest 
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name: SCRIPT 
type: string 

- default: ./Dockerfile 
name: CONTEXT_DIR 
type: string 

- default: 
name: IMAGE_DOCKERFILE 
type: string 

- default: 
name: IMAGE_CONTEXT_DIR 
type: string 


tasks: 

- name: fetch-repo 
params: 
- name: url 


value: $(params.GIT_REPO) 
- mame: revision 
value: $(params.GIT_REF) 
- name: deleteExisting 
value: "true" 
- name: verbose 
value: "true" 
taskRef: 
kind: Task 
name: git-clone 
workspaces: 
- name: output 
workspace: app-source 
- name: build-app 
params: 
- mame: GOALS 
value: 
- -DskipTests 
- clean 
- package 
name: CONTEXT_DIR 
value: $(params.CONTEXT_DIR) 
runAfter: 
- fetch-repo 
taskRef: 
kind: Task 
name: maven 
workspaces: 
- Name: maven-settings 
workspace: maven-settings 
- Mame: source 
workspace: app-source 
- name: build-push-image 
params: 
- mame: IMAGE 
value: $(params.DESTINATION_IMAGE) 
name: DOCKERFILE 
value: $(params.IMAGE_DOCKERFILE) 
name: CONTEXT 
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value: $(params.IMAGE_CONTEXT_DIR) 
runAfter: 
- build-app 
taskRef: 
kind: Task 
name: buildah 
workspaces: 
- Mame: source 
workspace: app-source 
- name: deploy 
params: 
- name: script 
value: $(params.SCRIPT) 
runAfter: 
- build-push-image 
taskRef: 
kind: Task 
name: kubernetes-actions 
workspaces: 
- name: app-source 
- name: maven-settings 


Let’s create the resource: 


kubectl create -f tekton-greeter-pipeline-hub. yaml 


We are using the same Secret and ServiceAccount defined in 
Recipe 6.5 to log in against Quay.io in order to push the container 
image. 


You can now start the Pipeline as follows: 


tkn pipeline start tekton-greeter-pipeline-hub \ 

--serviceaccount='tekton-deployer-sa' \ 

--param GIT_REPO='https://github.com/gitops-cookbook/tekton-tutorial- 
greeter.git' \ 

--param GIT_REF='master' \ 

--param CONTEXT_DIR='quarkus' \ 

--param DESTINATION_IMAGE='quay.io/gitops-cookbook/tekton-greeter:latest' \ 

--param IMAGE_DOCKERFILE='quarkus/Dockerfile' \ 

--param IMAGE_CONTEXT_DIR='quarkus' \ 

--param SCRIPT='kubectl create deploy tekton-greeter --image=quay.io/gitops- 
cookbook/tekton-greeter:latest' \ 

--workspace name=app-source,claimName=app-source-pvc \ 

--workspace name=maven-settings,emptyDir="" \ 

--use-param-defaults \ 

--showLlog 


[fetch-repo : clone] + CHECKOUT_DIR=/workspace/output/ 

[fetch-repo : clone] + /ko-app/git-init '-url=https://github.com/gitops-cookbook/ 
tekton-tutorial-greeter.git' '-revision=master' '-refspec=' '-path=/workspace/out 
put/' '-sslVerify=true' '-submodules=true' '-depth=1' '-sparseCheckoutDirectories=' 
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[fetch-repo : clone] {"level":"info","ts":1660819038.5526028,"caller":"git/ 
git.go:170","msg":"Successfully cloned https://github.com/gitops-cookbook/tekton- 
tutorial-greeter.git @ d9291c456db1ce29177b77ffeaa9b71ad80a50e6 (grafted, HEAD, ori 
gin/master) in path /workspace/output/"} 

[fetch-repo : clone] {"level":"info","ts":1660819038.5722632,"caller":"git/ 
git.go:208","msg":"Successfully initialized and updated submodules in path /work 
riba i cata "3 
[fetch-repo : clone] + cd /workspace/output/ 

[fetch- repo : clone] + git rev-parse HEAD 

[fetch-repo : clone] + RESULT_SHA=d9291c456db1ce29177b77ffeaa9b71ad80a50e6 
[fetch-repo : clone] + EXIT_CODE=0 

[fetch-repo : clone] + '[' 0 '!=' 0 ] 

[fetch-repo : clone] + printf '%s' d9291c456db1ce29177b77ffeaa9b71ad80a50e6 
[fetch-repo : clone] + printf '%s' https://github.com/gitops-cookbook/tekton- 
tutorial- grater, git 


[build-app : mvn-goals] [INFO] [org.jboss.threads] JBoss Threads version 
3.1.1.Final 
[build-app : mvn-goals] [INFO] [io.quarkus.deployment.QuarkusAugmentor ] Quarkus 
augmentation completed in 1296ms 

[build-app : mvn-goals] [INFO] 
[build-app : mvn-goals] [INFO] BUILD SUCCESS 

[build-app : mvn-goals] [INFO] 
[build-app : mvn-goals] [INFO] Total time: 03:18 min 

[build-app : mvn-goals] [INFO] Finished at: 2022-08-18T10:31:00Z 
[build-app : mvn-goals] [INFO] 
[build-push-image : build] STEP 1/2: FROM registry.access.redhat.com/ubi8/ 
openjdk-11 

[build-push-image : build] Trying to pull registry.access.redhat.com/ubi8/ 
openjdk-11:latest... 

[build-push-image : build] Getting image source signatures 

[build-push-image : build] Checking if image destination supports signatures 
[build-push-image : build] Copying blob 

sha256: €441d34134fac91baa79be3e2bb8fb3dba71baSc1ea012cbSdaeb7180a054687 
[build-push-image : build] Copying blob 

sha256: 1e09a5ee0038fbe06a18e7F355188bbabc387467144abcd435F7544fef395aal 
[build-push-image : build] Copying blob 
sha256:0d725b91398ed3db11249808d89e688e62e511bbd4a2e875ed8493ce1lfebdb2c 
[build-push-image : build] Copying blob 
sha256:e441d34134fac91baa79be3e2bb8fb3dba71ba5c1ea012cb5daeb7180a054687 
[build-push-image : build] Copying blob 

sha256: 1e09a5ee0038fbe06a18e7F355188bbabc387467144abcd435f7544fef395aal 
[build-push-image : build] Copying blob 
sha256:0d725b91398ed3db11249808d89e688e62e511bbd4a2e875ed8493ce1febdb2c 
[build-push-image : build] Copying config 

sha256: Sa al ec ree a ak lem a 
[build-push-image : build] Writing manifest to image destination 

[build- push-image : build] Storing signatures 

[build-push-image : build] STEP 2/2: COPY target/quarkus-app /deployments/ 
[build-push-image : build] COMMIT quay.io/gitops-cookbook/tekton-greeter: latest 
[build-push-image : build] --> c07e36a8e61 
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[build-push-image : build] Successfully tagged quay.io/gitops-cookbook/tekton- 
greeter: Latest 

[build-push-image : build] 
c07e36a8e6104d2e5c7d79a6cd34cd7b44eb093c39ef6c1487a37d7bd2305b8a 
[build-push-image : build] Getting image source signatures 
[build-push-image : build] Copying blob 

sha256: 7853a7797845542e3825d4f305e4784ea7bf492cd4364fc93b9afba3ac0c9553 
[build-push-image : build] Copying blob 

sha256: 8e0e04b5c700a86f4a112f41e7e767a9d7c539Fe3391611313bf 76edb07eeab1 
[build-push-image : build] Copying blob 
sha256:647a854c512bad44709221b6b0973e884f 29bcb5a380ee32e95bfb0189b620e6 
[build-push-image : build] Copying blob 
sha256:69c55192bed92cbb669c88eb3c36449b64ac93ae466abfff2a575273ce05a39e 
[build-push-image : build] Copying config 
sha256:c07e36a8e6104d2e5c7d79a6cd34cd7b44eb093c39ef6c1487a37d7bd2305b8a 
[build-push-image : build] Writing manifest to image destination 
[build-push-image : build] Storing signatures 

[build-push-image : build] 
sha256:12dd3deb6305b9e125309b68418d0bb81f805e0fe7ac93942dc94764aee9F492quay.io/ 
gitops-cookbook/tekton-greeter: latest 

[deploy : kubectl] deployment.apps/tekton-greeter created 


You can use the Tekton Dashboard to create and visualize your 
running Tasks and Pipelines as shown in Figure 6-8. 
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See Also 


e Tekton Catalog 


6.8 Using Tekton Triggers to Compile and Package an 
Application Automatically When a Change Occurs on Git 


Problem 


You want to automate your CI/CD Pipelines when a change on Git occurs. 


Solution 


Tekton Triggers is the Tekton component that brings automation for Tasks and 
Pipelines with Tekton. It is an interesting feature for a GitOps strategy for cloud 
native CI/CD as it supports external events from a large set of sources such as Git 
events (Git push or pull requests). 


Most Git repository servers support the concept of webhooks, calling to an external 
source via HTTP(S) when a change in the code repository happens. Tekton provides 
an API endpoint that supports receiving hooks from remote systems in order to trig- 
ger builds. By pointing the code repository’s hook at the Tekton resources, automated 
code/build/deploy pipelines can be achieved. 


The installation of Tekton Triggers, which we discussed in Recipe 6.1, brings a set of 
CRDs to manage event handling for Tasks and Pipelines. In this recipe we will use the 
following, as illustrated also in Figure 6-9: 


EventListener Pod 
Parameters Create Kubernetes 
ce(s 
Eent TriggerBinding TriggerTemplate Resource(s) 
Figure 6-9. Tekton Triggers 
TriggerTemplate 
A template for newly created resources. It supports parameters to create specific 
PipelineRuns. 
TriggerBinding 


Validates events and extracts payload fields. 


6.8 Using Tekton Triggers to Compile and Package an Application Automatically When a Change Occurs on Git | 135 


EventListener 
Connects TriggerBindings and TriggerTemplates into an addressable endpoint 
(the event sink). It uses the extracted event parameters from each Trigger 
Binding (and any supplied static parameters) to create the resources specified in 
the corresponding TriggerTempLate. It also optionally allows an external service 
to preprocess the event payload via the interceptor field. 


Before creating these resources, you need to set up permissions to let Tekton Triggers 
create Pipelines and Tasks. You can use the setup available from the book’s repository 
with the following command: 


kubectl apply \ 
-f https: //raw.githubusercontent.com/gitops-cookbook/chapters/main/chapters/ch06/ 
rbac.yaml 


This will create a new ServiceAccount named tekton-triggers-sa that has the per- 
missions needed to interact with the Tekton Pipelines component. As confirmation, 
from the previous command you should get the following output: 


serviceaccount/tekton-triggers-sa created 
rolebinding.rbac.authorization.k8s.io/triggers-example-eventlistener-binding con- 
figured 
clusterrolebinding.rbac.authorization.k8s.io/triggers-example-eventlistener- 
clusterbinding configured 


You can now add automation to your Pipelines like the one we defined in Recipe 6.7 
creating these three resources: 


apiVersion: triggers.tekton.dev/vialphat 
kind: TriggerTemplate 
metadata: 
name: tekton-greeter-triggertemplate 
spec: 
params: 
- name: git-revision 
- name: git-commit-message 
- name: git-repo-url 
- name: git-repo-name 
- name: content-type 
- name: pusher-name 
resourcetemplates: 
- apiVersion: tekton.dev/v1beta1 
kind: PipelineRun 
metadata: 
labels: 
tekton.dev/pipeline: tekton-greeter -pipeline-hub 
name: tekton-greeter -pipeline-webhook-$(uid) 
spec: 
params: 
- mame: GIT_REPO 
value: $(tt.params.git-repo-url) 
- mame: GIT_REF 
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value: $(tt.params.git-revision) 
serviceAccountName: tekton-triggers-example-sa 
pipeLineRef: 
name: tekton-greeter-pipeline-hub 
workspaces: 
- name: app-source 
persistentVolumeClain: 
claimName: app-source-pvc 
- name: maven-settings 
emptyDir: {} 


apiVersion: triggers.tekton.dev/vialphat 
kind: TriggerBinding 
metadata: 
Name: tekton-greeter-triggerbinding 
spec: 
params: 
- name: git-repo-url 
value: $(body.repository.clone_url) 
- name: git-revision 
value: $(body.after) 


apiVersion: triggers.tekton.dev/vialphat 
kind: EventListener 


metadata: 
name: tekton-greeter-eventlistener 
spec: 
serviceAccountName: tekton-triggers-example-sa 
triggers: 
- bindings: 
- ref: tekton-greeter-triggerbinding 
template: 


ref: tekton-greeter-triggertempLate 
You can create the resources just listed as follows: 


kubectl create -f tekton-greeter-triggertempLate. yaml 
kubectl create -f tekton-greeter-triggerbinding. yaml 
kubectl create -f tekton-greeter-eventlistener.yaml 


You should get the following output: 


triggertemplate.triggers.tekton.dev/tekton-greeter-triggertemplate created 
triggerbinding.triggers.tekton.dev/tekton-greeter-triggerbinding created 
eventlistener.triggers.tekton.dev/tekton-greeter-eventlistener created 


Contextually, a new pod is created representing the EventListener: 
kubectl get pods 
You should get output similar to the following: 


NAME READY STATUS RESTARTS AGE 
el-tekton-greeter-eventlistener -5db7b9fcf9-6nrgx 1/1 Running 0 10s 


The EventListener pod listens for events at a specified port, and it is bound to a 
Kubernetes Service: 
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kubectl get svc 


You should get output similar to the following: 


NAME TYPE CLUSTER-IP EXTERNAL-IP& 
PORT(S) AGE 
el-tekton-greeter-eventlistener ClusterIP 10.100.36.199 <none> b 


8080/TCP,9000/TCP 10s 


If you are running your Git server outside the cluster (e.g., GitHub or GitLab), 
you need to expose the Service, for example, with an Ingress. Afterwards you can 
configure webhooks on your Git server using the EventListener URL associated to 
your Ingress. 


With Minikube you can add support for Ingresses with this com- 
mand: minikube addons enable ingress. Then you need to map 
a hostname for the Ingress. 


For the purpose of this book we can just simulate the webhook as it would come from 
the Git server. 


First you can map the EventListener Service to your local networking with the 
following command: 


kubectl port-forward svc/el-tekton-greeter-eventlistener 8080 


Then you can invoke the Trigger by making an HTTP request to http://localhost:8080 
using curl. The HTTP request must be a POST request containing a JSON payload 
and it should contain the fields referenced via a TriggerBinding. In our case we 
mapped body.repository.clone_url and body.after. 


Check the documentation of your Git server to get the list of 
parameters that a webhook can generate. In this example we are 
using the GitHub Webhooks reference. 


To test Triggers, run this command: 


curl -X POST \ 

http://localhost:8080 \ 

-H 'Content-Type: application/json' \ 

-d '{ "after": "d9291c456db1ce29177b77ffeaa9b71ad80a50e6", "repos 
itory": { "clone_url" : "https://github.com/gitops-cookbook/tekton-tutorial- 
greeter.git" } }' 


You should get output similar to the following: 


138 | Chapter 6: Cloud Native CI/CD 


{"eventListener": "tekton-greeter-eventlistener","namespace":"default","eventListe 
nerUID":"c00567eb-d798-4c4a-946d-f1732fdfc313", "eventID":"17dd25bb-alfe-4f84-8422- 
c3abc5f10066"} 


A new Pipeline now is started and you can check it with the following command: 
tkn pipelinerun ls 

You should see it in Running status as follows: 
tekton-greeter - pipeline -3244b67f -31d3-4597-afic-3c1aa6693719 4 seconds ago 


--- Running 


See Also 


e Tekton Triggers examples 
e Getting Started with Tekton Triggers 


e Securing webhooks with event listeners 


6.9 Update a Kubernetes Resource Using Kustomize and 
Push the Change to Git 


Problem 


You want to use Kustomize in your Tekton Pipelines in order to automate Kubernetes 
manifests updates. 


Solution 


As we discussed in Chapter 4, Kustomize is a powerful tool to manage Kubernetes 
manifests. Kustomize can add, remove, or patch configuration options without fork- 
ing. In Recipe 4.2 you saw how to update a Kubernetes Deployment with a new 
container image hash using the kustomize CLI. 


In this recipe, you'll see how to let Tekton update it using Kustomize. This is very 
useful for GitOps as it allows an automated update on Git to the manifests describing 
an application running on Kubernetes, favoring the interconnection with a GitOps 
tool such as Argo CD in order to sync resources (see Chapter 7). 


When adopting the GitOps approach, its common to have one or more repositories 
for the Kubernetes manifests and then one or more repositories for the apps as well. 


Thus lets introduce a Task that accepts the Kubernetes manifests repository as a 
parameter and can update the container image reference as seen in Recipe 4.2: 
apiVersion: tekton.dev/vibeta1 


kind: Task 
metadata: 
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annotations: 
tekton.dev/pipelines.minVersion: 0.12.1 
tekton.dev/tags: git 
name: git-update-deployment 
labels: 
app.kubernetes.io/version: '0.2' 
operator.tekton.dev/provider-type: community 
spec: 
description: >- 
This Task can be used to update image digest in a Git repo using kustomize. 
It requires a secret with credentials for accessing the git repo. 
params: 
- mame: GIT_REPOSITORY 
type: string 
- name: GIT_REF 
type: string 
- mame: NEW_IMAGE 
type: string 
- mame: NEW _DIGEST 
type: string 
- mame: KUSTOMIZATION_PATH 
type: string 
results: 
- description: The commit SHA 
name: commit 
steps: 
- image: 'docker.io/alpine/git:v2.26.2' 
name: git-clone 
resources: {} 
script: > 
rm -rf git-update-digest-workdir 


git clone $(params.GIT_REPOSITORY) -b $(params.GIT_REF) 
git-update-digest-workdir 

workingDir: $(workspaces.workspace. path) 

image: '‘quay.io/wpernath/kustomize-ubi:latest' 

name: update-digest 

resources: {} 

script: > 
cd git-update-digest-workdir/$(params .KUSTOMIZATION_PATH) 


kustomize edit set image $(params.NEW_IMAGE)@$(params.NEW_DIGEST) 


echo "AEH HHH HH HH HH HH HH HH HH " 
echo "### kustomization.yaml ###" 
echo "#AHEHHH HHH HHH HH HH HH HH HH " 
cat kustomization.yaml 
workingDir: $(workspaces.workspace. path) 


- image: 'docker.io/alpine/git:v2.26.2' 
Name: git-commit 
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resources: {} 
script: | 
cd git-update-digest-workdir 


git config user.email "tektonbot@redhat.com" 
git config user.name "My Tekton Bot" 


git status 
git add $(params.KUSTOMIZATION_PATH)/kustomization. yaml 
git commit -m "[ci] Image digest updated" 


git push 


RESULT_SHA="$(git rev-parse HEAD | tr -d '\n')" 
EXIT_CODE="$?" 
if [ "SEXIT_CODE" != 0 ] 
then 
exit SEXIT_CODE 
fi 
# Make sure we don't add a trailing newline to the result! 
echo -n "SRESULT_SHA" > $(results.commit.path) 
workingDir: $(workspaces.workspace. path) 
workspaces: 
- description: The workspace consisting of maven project. 
name: workspace 


This Task is composed of three steps: 


git-clone 
Clones the Kubernetes manifests repository 


update-digest 
Runs kustomize to update the Kubernetes Deployment with a container image 
hash given as a parameter 


git-commit 
Updates the Kubernetes manifest repo with the new container image hash 


You can create the Task with the following command: 
kubectl create -f git-update-deployment-task.yaml 
You should get the following output: 
task. tekton.dev/git-update-deployment created 


You can now add this Task to a Pipeline similar to the one you saw in Recipe 6.7 in 
order to automate the update of your manifests with Kustomize: 


apiVersion: tekton.dev/vibeta1 
kind: Pipeline 
metadata: 
Name: pacman-pipeline 
spec: 
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params: 


default: https: //github.com/gitops-cookbook/pacman-kikd.git 
name: GIT_REPO 

type: string 

default: master 

name: GIT_REVISION 

type: string 

default: quay.io/gitops-cookbook/pacman-kikd 

name: DESTINATION_IMAGE 

type: string 

default: 

name: CONTEXT_DIR 

type: string 

default: ‘https://github.com/gitops-cookbook/pacman-kikd-manifests.git' 
name: CONFIG_GIT_REPO 

type: string 

default: main 

name: CONFIG_GIT_REVISION 

type: string 


tasks: 
name: fetch-repo 
params: 
- name: url 


value: $(params.GIT_REPO) 
- name: revision 
value: $(params.GIT_REVISION) 
- name: deleteExisting 
value: "true" 
taskRef: 
name: git-clone 
workspaces: 
- name: output 
workspace: app-source 
name: build-app 
taskRef: 
name: maven 
params: 
- name: GOALS 
value: 
- -DskipTests 
- clean 
- package 
- mame: CONTEXT_DIR 
value: "$(params.CONTEXT_DIR)" 
workspaces: 
- name: maven-settings 
workspace: maven-settings 
- name: source 
workspace: app-source 
runAfter: 
- fetch-repo 
name: build-push-image 
taskRef: 
name: buildah 
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params: 
- mame: IMAGE 
value: "S(params.DESTINATION_IMAGE)" 
workspaces: 
- name: source 
workspace: app-source 
runAfter: 
- build-app 
- name: git-update-deployment 
params: 
- mame: GIT_REPOSITORY 
value: $(params.CONFIG_GIT_REPO) 
- mame: NEW_IMAGE 
value: $(params.DESTINATION_IMAGE) 
- mame: NEW _DIGEST 
value: $(tasks.build-push-image.results.IMAGE_DIGEST) 1] 
- mame: KUSTOMIZATION_PATH 
value: env/dev 
- mame: GIT_REF 
value: $(params.CONFIG_GIT_REVISION) 
runAfter: 
- build-push-image 
taskRef: 
kind: Task 
name: git-update-deployment 
workspaces: 
- name: workspace 
workspace: app-source 
workspaces: 
- name: app-source 
- mame: maven-settings 


@ As you can see from this example, you can take a result of a previous Task as 
an input for the following one. In this case the hash of the container image 
generated by the build-push-image Task is used to update the manifests with 
Kustomize. 


You can create the Pipeline with the following command: 
kubectl create -f pacman-pipeline.yaml 

You should get the following output: 
pipeline. tekton.dev/pacman-pipeline created 


The git-commit step requires authentication to your Git server in order to push the 
updates to the repo. Since this example is on GitHub, we are using a GitHub Personal 
Access Token (see Recipe 6.4) attached to the ServiceAccount tekton-bot-sa. 
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Make sure to add the repo and registry’s Kubernetes Secrets as described in Recipes 
6.4 and 6.5: 


kubectl patch serviceaccount tekton-bot-sa -p '{"secrets": [{"name": "git- 
secret"}]}' 
kubectl patch serviceaccount tekton-bot-sa \ 
-p '{"secrets": [{"name": "containerregistry- 
secret"}]}' 


Make sure you have created a PVC for the Pipeline as defined in 
Recipe 6.7. 


Now you can start the Pipeline as follows: 


tkn pipeline start pacman-pipeline \ 
--serviceaccount='tekton-bot-sa' \ 
--param GIT_REPO='https://github.com/gitops-cookbook/pacman-kikd.git' \ 
--param GIT_REVISION='main' \ 
--param DESTINATION_IMAGE='quay.io/gitops-cookbook/pacman-kikd:latest' \ 
--param CONFIG_GIT_REPO='https://github.com/gitops-cookbook/pacman-kikd- 
manifests.git' \ 
--param CONFIG_GIT_REVISION='main' \ 
--workspace name=app-source,claimName=app-source-pvc \ 
--workspace name=maven-settings,emptyDir="" \ 
--use-param-defaults \ 
--showLlog 


6.10 Update a Kubernetes Resource Using Helm and 
Create a Pull Request 


Problem 


You want to automate the deployment of Helm-packaged apps with a Tekton 
Pipeline. 


Solution 


In Chapter 5 we discussed Helm and how it can be used to manage applications 
on Kubernetes in a convenient way. In this recipe you'll see how to automate Helm- 
powered deployments through a Pipeline in order to install or update an application 
running on Kubernetes. 


As shown in Recipe 6.7, you can use Tekton Hub to find and install Tekton Tasks. In 
fact, you can use the helm-upgrade-from-repo Task to have Helm support for your 
Pipelines. 
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To install it, run this command: 
tkn hub install task helm-upgrade-from-repo 


This Task can install a Helm Chart from a Helm repository. For this example, we pro- 
vide a Helm repository in this book’s repository that you can add with the following 
command: 


helm repo add gitops-cookbook https://gitops-cookbook.github.io/helm-charts/ 
You should get the following output: 

"gitops-cookbook" has been added to your repositories 
You can install the Helm Chart with the following command: 

helm install pacman gitops-cookbook/pacman 
You should get output similar to the following: 


NAME: pacman 

LAST DEPLOYED: Mon Aug 15 17:02:21 2022 
NAMESPACE: default 

STATUS: deployed 

REVISION: 1 

TEST SUITE: None 

USER-SUPPLIED VALUES: 


{} 


The app should be now deployed and running on Kubernetes: 
kubectl get pods -l=app.kubernetes.io/name=pacman 
You should get the following output: 


NAME READY STATUS RESTARTS AGE 
pacman-6798d65d84-9mt8p 1/1 Running 0 30s 


Now let’s update the Deployment with a Tekton Task running a helm upgrade with 
the following TaskRun: 


apiVersion: tekton.dev/vibeta1 
kind: TaskRun 


metadata: 
generateName: helm-pacman-run- 
spec: 
serviceAccountName: tekton-deployer-sa (13 
taskRef: 
name: helm-upgrade-from-repo 
params: 


- name: helm_repo 
value: https://gitops-cookbook.github.io/helm-charts/ 
- name: chart_name 
value: gitops-cookbook/pacman 
- name: release_version 
value: 0.1.0 
- name: release_name 


6.10 Update a Kubernetes Resource Using Helm and Create a Pull Request | 145 


value: pacman 
- name: overwrite. 
value: replicaCc 


values 
ount=2 @ 


@ The helm-upgrade-from-repo Task needs permission to list objects in the work- 
ing namespace, so you need a ServiceAccount with special permissions as seen 


(2) 


in Recipe 6.6. 


You can override 


values in the Charts values.yaml file by adding them in this 


param. Here we are setting up two replicas for the Pac-Man game. 


Run the Task with the following command: 


kubectl create -f h 


elm-pacman-taskrun.yaml 


You should get output similar to the following: 


taskrun.tekton.dev/ 


helm-pacman-run-qghx8 created 


Check logs with tkn CLI and select the running Task: 


tkn taskrun logs -f 


You should get output similar to the following, where you can see the Helm upgrade 
has been successfully performed: 


[upgrade-from-repo | 
[upgrade-from-repo | 
TED 

VERSION 
[upgrade-from-repo | 
17:02:21.633934129 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo | 
ries... 
[upgrade-from-repo | 
repository 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo | 
Service "pacman" 
[upgrade-from-repo | 
out of 5m@s 
[upgrade-from-repo] 
1 out of 2 expected 
[upgrade-from-repo] 


current installed helm releases 


NAME NAMESPACE REVISION UPDA- 
STATUS CHART APP 
pacman default 1 2022-08-15 
+0200 +0200 deployed pacman-0.1.0 1.0.0 


parsing helms repo name... 

adding helm repo... 

"gitops-cookbook" has been added to your repositories 

adding updating repo... 

Hang tight while we grab the latest from your chart reposito- 


...Successfully got an update from the "gitops-cookbook" chart 


Update Complete. Happy Helming! # 

installing helm chart... 

history.go:56: [debug] getting history for release pacman 
upgrade.go:123: [debug] preparing upgrade for pacman 
upgrade.go:131: [debug] performing update for pacman 
upgrade.go:303: [debug] creating upgraded release for pacman 
client.go:203: [debug] checking 2 resources for changes 
client.go:466: [debug] Looks like there are no changes for 


wait.go:47: [debug] beginning wait for 2 resources with time- 
ready.go:277: [debug] Deployment is not ready: default/pacman. 


pods are ready 
ready.go:277: [debug] Deployment is not ready: default/pacman. 
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1 out of 2 expected 
[upgrade-from-repo | 
1 out of 2 expected 
[upgrade-from-repo | 
for pacman 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo] 
[upgrade-from-repo | 
[upgrade-from-repo | 


pods are ready 


ready.go:277: [debug] Deployment is not ready: default/pacman. 


pods are ready 


upgrade.go:138: [debug] updating status for upgraded release 


Release "pacman" has been upgraded. Happy Helming! 
NAME: pacman 

LAST DEPLOYED: Mon Aug 15 15:23:31 2022 

NAMESPACE: default 

STATUS: deployed 

REVISION: 2 

TEST SUITE: None 

USER-SUPPLIED VALUES: 

replicaCount: 2 


COMPUTED VALUES: 
image: 
containerPort: 8080 
pullPolicy: Always 
repository: quay.io/gitops-cookbook/pacman-kikd 
tag: 1.0.0 
replicaCount: 2 
securityContext: {} 


HOOKS: 
MANIFEST: 
# Source: pacman/templates/service. yanl 
apiVersion: v1 
kind: Service 
metadata: 
labels: 
app.kubernetes.io/name: pacman 
name: pacman 


spec: 
ports: 
- name: http 
port: 8080 
targetPort: 8080 
selector: 


app.kubernetes.io/name: pacman 
# Source: pacman/templates/deployment.yaml 
apiVersion: apps/v1 
kind: Deployment 
metadata: 
name: pacman 
labels: 
app.kubernetes.io/name: pacman 
app.kubernetes.io/version: "1.0.0" 


spec: 
replicas: 2 
selector: 
matchLabels: 
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[upgrade-from-repo] app.kubernetes.io/name: pacman 
[upgrade-from-repo] template: 

[upgrade-from-repo | metadata: 

[upgrade-from-repo] labels: 

[upgrade-from-repo | app.kubernetes.io/name: pacman 
[upgrade-from-repo | spec: 

[upgrade-from-repo] containers: 

[upgrade-from-repo] - image: "quay.io/gitops-cookbook/pacman-kikd:1.0.0" 
[upgrade-from-repo] imagePullPolicy: Always 
[upgrade-from-repo] securityContext: 
[upgrade-from-repo | {} 

[upgrade-from-repo] Name: pacman 
[upgrade-from-repo | ports: 

[upgrade-from-repo | - containerPort: 8080 
[upgrade-from-repo] name: http 
[upgrade-from-repo | protocol: TCP 
[upgrade-from-repo] 


kubectl get deploy -l=app.kubernetes.io/name=pacman 


pacman 2/2 2 2 9s 


6.11 Use Drone to Create a Pipeline for Kubernetes 


Problem 


You want to create a CI/CD pipeline for Kubernetes with Drone. 


Solution 


Drone is an open source project for cloud native continuous integration (CI). It uses 
YAML build files to define and execute build pipelines inside containers. 


It has two main components: 


Server 
Integrates with popular SCMs such as GitHub, GitLab, or Gitea 


Runner 
Acts as an agent running on a certain platform 


You can install the Server of your choice following the documentation and install the 
Kubernetes Runner. 


In this example you will create a Java Maven-based pipeline using the Pac-Man app. 
First, install the Drone CLI for your OS; you can get it from the official website here. 
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On macOS, drone is available through Homebrew as follows: 


brew tap drone/drone && brew install drone 


Then configure Drone, copy the DRONE_TOKEN from your instance under the Drone 
Account settings page, then create/update the file called .envrc.local and add the 
variables to override: 


export DRONE_TOKEN="<YOUR-TOKEN>" 
Ensure the token is loaded: 
drone info 
Now activate the repo in Drone: 
drone repo enable https://github.com/gitops-cookbook/pacman-kikd.git 


Similarly to Tekton, Drone’s pipeline will compile, test, and build the app. Then it will 
create and push the container image to a registry. 


Add credentials to your container registry as follows (here, were using Quay.io): 


drone secret add --name image_registry \ 
--data quay.io https://github.com/gitops-cookbook/pacman-kikd.git 


drone secret add --name image_registry_user \ 
--data YOUR_REGISTRY_USER https://github.com/gitops-cookbook/pacman-kikd.git 


drone secret add --name image_registry_password \ 
--data YOUR_REGISTRY_PASS https://github.com/gitops-cookbook/pacman-kikd.git 


drone secret add --name destination_image \ 
--data quay.io/YOUR_REGISTRY_USER>/pacman-kikd.git https://github.com/gitops- 
cookbook/pacman-kikd.git 


Create a file called .drone.yaml as follows: 


kind: pipeline 
type: docker 
name: java-pipeline 
platform: 
os: linux 
arch: arm64 
trigger: 
branch: 
- main 
clone: 
disable: true 
steps: 
- name: clone sources 
image: alpine/git 
pull: if-not-exists 
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commands: 
- git clone https://github.com/gitops-cookbook/pacman-kikd.git . 
- git checkout $DRONE_COMMIT 
- name: maven-build 
image: maven:3-jdk-11 
commands: 
- mvn install -DskipTests=true -B 
- mvn test -B 
- name: publish 
image: plugins/docker:20.13 
pull: if-not-exists 
settings: 
tags: "Latest" 
dockerfile: Dockerfile 
insecure: true 
mtu: 1400 
username: 
from_secret: image_registry_user 
password: 
from_secret: image_registry_password 
registry: 
from_secret: image_registry 
repo: 
from_secret: destination_image 


Start the pipeline: 


drone exec --pipeline=java-pipeline 


You can also trigger the pipeline to start by pushing to your Git 
repo. 


See Also 


e Example Maven Pipeline from Drone docs 


e Complete Quarkus pipeline example in Drone 


6.12 Use GitHub Actions for Cl 


Problem 


You want to use GitHub Actions for CI in order to compile and package an app as a 
container image ready to be deployed in CD. 
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Solution 


GitHub Actions are event-driven automation tasks available for any GitHub reposi- 
tory. An event automatically triggers the workflow, which contains a job. The job 
then uses steps to control the order in which actions are run. These actions are the 
commands that automate software building, testing, and deployment. 


In this recipe, you will add a GitHub Action for building the Pac-Man game container 
image, and pushing it to the GitHub Container Registry. 


As GitHub Actions are connected to repositories, you can fork the 
Pac-Man repository from this book’s code repositories to add your 
GitHub Actions. See the documentation about forking repositories 
for more info on this topic. 


GitHub Actions workflows run into environments and they can reference an environ- 
ment to use the environment’s protection rules and secrets. 


Workflows and jobs are defined with a YAML file containing all the needed 
steps. Inside your repository, you can create one with the path .github/workflows/ 
pacman-ci-action.yml: 


# This is a basic workflow to help you get started with Actions 


Name: pacman-ci-action (1 


env: © 
IMAGE_REGISTRY: ghcr.io/${{ github.repository_owner }} 
REGISTRY_USER: ${{ github.actor }} 
REGISTRY_PASSWORD: ${{ github.token }} 
APP_NAME: pacman 
IMAGE_TAGS: 1.0.0 ${{ github.sha }} 


# Controls when the workflow will run 
on: 
# Triggers the workflow on push or pull request events but only for the 
# "main" branch 
push: © 
branches: [ "main" ] 
pull_request: 
branches: [ "main" ] 


# Allows you to run this workflow manually from the Actions tab 
workflow_dispatch: 


# A workflow run is made up of one or more jobs that can run sequentially or in 
# parallel 
jobs: 

# This workflow contains a single job called "build-and-push" 

build-and-push: (4) 
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# The type of runner that the job will run on 
runs-on: ubuntu-latest 


# Steps represent a sequence of tasks that will be executed as part of the 
# job 
steps: 

# Checks-out your repository under $GITHUB_WORKSPACE, so your job can 

# access it 

- uses: actions/checkout@v3 


- mame: Set up JDK 11 
uses: actions/setup-java@v3 
with: 
java-version: '11' 
distribution: ‘adopt' 
cache: maven 


- name: Build with Maven 
run: mvn --batch-mode package 


- name: Buildah Action @ 
id: build-image 
uses: redhat-actions/buildah-build@v2 
with: 
image: ${{ env. IMAGE_REGISTRY }}/${{ env.APP_NAME }} 
tags: ${{ env. IMAGE_TAGS }} 
containerfiles: | 
. /Dockerfile 
- name: Push to Registry @ 
id: push-to-registry 
uses: redhat-actions/push-to-registry@v2 
with: 
image: ${{ steps.build-image.outputs.image }} 
tags: ${{ steps.build-image.outputs.tags }} 
registry: ${{ env. IMAGE_REGISTRY }} 
username: ${{ env.REGISTRY_USER }} 
password: ${{ env.REGISTRY_PASSWORD }} 


@ Name of the Action. 


© Environment variables to be used in the workflow. This includes default environ- 
ment variables and the Secret you added to the environment. 


© Heres where you define which type of trigger you want for this workflow. In this 
case, any change to the repository (Push) to the master branch will trigger the 
action to start. Check out the documentation for a full list of triggers that can be 
used. 


Name of this Job. 


© List of steps; each step contains an action for the pipeline. 
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© Buildah Build. This action builds container images using Buildah. 


@ Push to Registry. This action is used to push to the GitHub Registry using 
built-in credentials available for GitHub repository owners. 


After each Git push or pull request, a new run of the action is performed as shown in 
Figure 6-10. 


GitHub offers its own container registry available at ghcr.io, and 
container images are referenced as part of the GitHub Packages. 
By default the images are public. See this book’s repository as a 
reference. 


M Summary build 
n Q Search logs 
succeeded 3 hours ago in 1m 3s 
Jobs 
© build @ Set up job 


Run actions/checkout@v3 


Set up JDK 11 


Build with Maven 


Buildah Action 


Push to Registry 


Post Set up JDK 11 


Post Run actions/checkout@v3 


Complete job 


Figure 6-10. GitHub Actions Jobs 


See Also 


e GitHub Actions Jobs 
e Red Hat Actions 


e Deploy to Kubernetes cluster Action 
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CHAPTER 7 
Argo CD 


In the previous chapter, you learned about Tekton and other engines such as GitHub 
Actions to implement the continuous integration (CI) part of a project. 


Although CI is important because it’s where you build the application and check 
that nothing has been broken (running unit tests, component tests, etc.), there is 
still a missing part: how to deploy this application to an environment (a Kubernetes 
cluster) using the GitOps methodology and not creating a script running kubect1/ 
helm commands. 


As Daniel Bryant puts it, “If you weren't using SSH in the past to deploy your 
application in production, don’t use kubect1 to do it in Kubernetes.” 


In this chapter, we'll introduce you to Argo CD, a declarative, GitOps continuous 
delivery (CD) tool for Kubernetes. In the first part of the section, we'll see the 
deployment of an application using Argo CD (Recipes 7.1 and 7.2). 


Argo CD not only supports the deployment of plain Kubernetes manifests, but also 
the deployment of Kustomize projects (Recipe 7.3) and Helm projects (Recipe 7.4). 


A typical operation done in Kubernetes is a rolling update to a new version of the 
container, and Argo CD integrates with another tool to make this process smooth 
(Recipe 7.5). 


Delivering complex applications might require some orchestration on when and how 
the application must be deployed and released (Recipes 7.7 and 7.8). 


Well see how to: 


e Install and deploy the first application. 


e Use automatic deployment and self-healing applications. 
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e Execute a rolling update when a new container is released. 


e Give an order on the execution. 


In this chapter, we are using the https://github.com/gitops-cookbook/gitops- 
cookbook-sc.git GitHub repository as source directory. To run it successfully in this 
chapter, you should fork it and use it in the YAML files provided in the examples. 


7.1 Deploy an Application Using Argo CD 


Problem 
You want Argo CD to deploy an application defined in a Git repository. 


Solution 
Create an Application resource to set up Argo CD to deploy the application. 


To install Argo CD, create the argocd namespace and apply the Argo CD installation 
manifest: 


kubectl apply -n argocd \ 
-f https: //raw.githubusercontent.com/argoproj/argo-cd/v2.3.4/manifests/install.yaml 


Optional Steps 


It's not mandatory to install the Argo CD CLI tool, or expose the Argo CD server 
service to access the Argo CD Dashboard. Still, in this book, well use them in the 
recipes to show you the final result after applying the manifests. So, although not 
mandatory, we encourage you to follow the next steps to be aligned with the book. 


To install the argocd CLI tool, go to the Argo CD CLI GitHub release page and in the 
Assets section, download the tool for your platform. 


After installing the argocd tool, the argocd-server Kubernetes Service needs to 
be exposed. You can use any technique such as Ingress or set the service as LoadBa 
lancer but well use the kubectl port-forwarding to connect to the API server 
without exposing the service: 


kubectl port-forward svc/argocd-server -n argocd 9090:443 
At this point, you can access the Argo CD server using http://localhost:9090. 


The initial password for the admin account is generated automatically in a secret 
named argocd-initial-admin-secret in the argocd namespace: 


argoPass=$(kubectl -n argocd get secret argocd-initial-admin-secret -o json 
path="{.data.password}" | base64 -d) 
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argoURL=Localhost:9090 


argocd login --insecure --grpc-web SargoURL --username admin --password Sargo 
Pass 


‘admin: login' logged in successfully 


You should use the same credentials to access the Argo CD UI. 


Lets make Argo CD deploy a simple web application showing a box with a config- 
ured color. The application is composed of three Kubernetes manifest files, including 
a Namespace, a Deployment, and a Service definition. 


The files are located in the ch07/bgd folder of the book’s repository. 


All these files are known as an Application in Argo CD. Therefore, you must define 
it as such to apply these manifests in your cluster. 


Lets check the Argo CD Application resource file used for deploying the 
application: 


apiVersion: argoproj.io/vialpha1 
kind: Application 
metadata: 
Name: bgd-app 
Namespace: argocd 1] 
spec: 
destination: @ 
Namespace: bgd 
server: https://kubernetes.defauLlt.svc 
project: default © 
source: 
repoURL: https://github.com/gitops-cookbook/gitops-cookbook-sc.git (4) 
path: ch07/bgd @ 
targetRevision: main 


Namespace where Argo CD is installed 

Target cluster and namespace 

Installing the application in Argo CD’s default project 
The manifest repo where the YAML resides 


The path to look for manifests 


O © © 8 8 8 


Branch to checkout 
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In the terminal window, run the following command to register the Argo CD 
application: 


kubectl apply -f manual-bgd-app.yaml 


At this point, the application is registered as an Argo CD application. 


You can check the status using either argocd or the UI; run the following command 
to list applications using the CLI too: 


argocd app list 
And the output is something like: 


NAME CLUSTER NAMESPACE PROJECT STATUS 
bgd-app https://kubernetes.default.svc bgd default OutOfSync 


The important field here is STATUS. It’s OutOfSync, which means the application is 
registered, and there is a drift between the current status (in this case, no application 
deployed) and the content in the Git repository (the application deployment files). 


You'll notice that no pods are running if you get all the pods from the bgd namespace: 


kubectl get pods -n bgd 


No resources found in bgd namespace. 


Argo CD doesn’t synchronize the application automatically by default. It just shows a 
divergence, and the user is free to fix it by triggering a synchronized operation. 


With the CLI, you synchronize the application by running the following command in 
a terminal: 


argocd app sync bgd-app 


And the ouput of the command shows all the important information regarding the 
deployment: 


Name: bgd-app 

Project: default 

Server: https://kubernetes.default.svc 

Namespace: bgd 

URL: https://openshift-gitops-server-openshift-gitops.apps.open- 
shift.sotogcp.com/applications/bgd-app 

Repo: https://github.com/lordofthejars/gitops-cookbook-sc.git 
Target: main 

Path: ch07/bgd 

SyncWindow: Sync Allowed 

Sync Policy: <none> 

Sync Status: Synced to main (384cd3d) 

Health Status: Progressing 

Operation: Sync 

Sync Revision: 384cd3d21c534e75cb6b1a6921a6768925b81244 

Phase: Succeeded 
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Start: 2022-06-16 14:45:12 +0200 CEST 


Finished: 2022-06-16 14:45:13 +0200 CEST 
Duration: 1s 
Message: successfully synced (all tasks run) 
GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE 

Namespace bgd bgd Running Synced Namespace/bgd cre- 
ated 

Service bgd bgd Synced Healthy service/bgd created 
apps Deployment bgd bgd Synced Progressing deploy- 


ment.apps/bgd created 


Namespace bgd Synced 


You can synchronize the application from the UI as well, by clicking the SYNC button 


as shown in Figure 7-1. 


Applications 


Project: 


SYNC STATUS Labels: 


Status: 
© O Unknown 
Reposito... 


D @ Synced Target R... 


OC © OutofSync Fath: 
Destinati... 


Namesp... 
HEALTH STATUS 


[O @ Unknown 
OC O Progressing 
L O Suspended 
O Y Healthy 

O Y Degraded 
OC @ Missing 


(_] FAVORITES ONLY 
© bgd-app 


default 


@ Healthy @ OutOfSync 
https://github.com/lordofthejars/gitops-... 
main 

ch07/bgd 

in-cluster 

bgd 


© DELETE 


Figure 7-1. Argo CD web console 
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If you get all the pods from the bgd namespace, you'll notice one pod running: 


kubectl get pods -n bgd 


NAME READY STATUS RESTARTS AGE 
bgd-788cb756f7-jll9n 1/1 Running 0 60s 


And the same for the Service: 


kubectl get services -n bgd 


NAME TYPE CLUSTER-IP EXTERNAL - IP PORT(S) 
bgd ClusterIP 172.30.35.199 <none> 8080:32761/TCP (1) 


@ Exposed port is 32761 


In the following sections, you'll need to access the deployed service to validate that 
it’s deployed. There are several ways to access services deployed to Minikube; for the 
following chapters, we use the Minikube IP and the exposed port of the service. 


Run the following command in a terminal window to get the Minikube IP: 


minikube ip -p gitops 
192.168.59.100 


Open a browser window, set the previous IP followed by the exposed port (in this 
example 192.168.59.100:32761), and access the service to validate that the color of 
the circles in the box is blue, as shown in Figure 7-2. 


B 
e 


ck 


Figure 7-2. Deployed application 
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Discussion 


Now it’s time to update the application deployment files. This time we will change the 
value of an environment variable defined in the bgd-deployment.yaml file. 


Open ch07/bgd/bgd-deployment.yaml in your file editor and change the COLOR envi- 
ronment variable value from blue to green: 
spec: 
containers: 
- image: quay.io/redhatworkshops/bgd: latest 
name: bgd 
env: 
- name: COLOR 
value: "green" 
In a terminal run the following commands to commit and push the file so the change 
is available for Argo CD: 


git add . 
git commit -m "Updates color" 


git push origin main 
With the change pushed, check the status of the application again: 


argocd app list 


NAME CLUSTER NAMESPACE PROJECT STATUS 
bgd-app https://kubernetes.default.svc bgd default Sync 


We see the application status is Sync. This happens because Argo CD uses a polling 
approach to detect divergences between whats deployed and what's defined in Git. 
After some time (by default, it’s 3 minutes), the application status will be OutOfSync: 


argocd app list 
NAME CLUSTER NAMESPACE PROJECT STATUS HEALTH 
bgd-app https://kubernetes.default.svc bgd default OutOfSync Healthy 


To synchronize the changes, run the sync subcommand: 
argocd app sync bgd-app 


After some seconds, access the service and validate that the circles are green, as 
shown in Figure 7-3. 
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D o ® 


© 


Figure 7-3. Deployed application 


To remove the application, use the CLI tool or the UI: 
argocd app delete bgd-app 


Also, revert the changes done in the Git repository to get the initial version of the 
application and push them: 


git revert HEAD 


git push origin main 


7.2 Automatic Synchronization 


Problem 


You want Argo CD to automatically update resources when there are changes. 


Solution 
Use the syncPolicy section with an automated policy. 


Argo CD can automatically synchronize an application when it detects differences 
between the manifests in Git and the Kubernetes cluster. 


A benefit of automatic sync is that there is no need to log in to the Argo CD API, 
with the security implications that involves (managing secrets, network, etc.), and the 
use of the argocd tool. Instead, when a manifest is changed and pushed to the Git 
repository with the changes to the tracking Git repo, the manifests are automatically 
applied. 


Let's modify the previous Argo CD manifest file (Application), adding the sync 
Policy section, so changes are deployed automatically: 
apiVersion: argoproj.io/vialpha1 


kind: Application 
metadata: 
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name: bgd-app 
Namespace: argocd 
spec: 
destination: 
Namespace: bgd 
server: https://kubernetes.defauLlt.svc 
project: default 
source: 
path: ch07/bgd 
repoURL: https://github.com/gitops-cookbook/gitops-cookbook-sc.git 
targetRevision: main 
syncPolicy: (13 
automated: {} (2) 


@ Starts the synchronization policy configuration section 


@ Argo CD automatically syncs the repo 


At this point, we can apply the Application file into a running cluster by running the 
following command: 


kubectl apply -f bgd/bgd-app.yaml 
Now, Argo CD deploys the application without executing any other command. 


Run the kubectl command or check in the Argo CD UI to validate that the deploy- 
ment is happening: 


kubectl get pods -n bgd 


NAME READY STATUS RESTARTS AGE 
bgd-788cb756f7-jll9n 1/1 Running 0 60s 


Access the service and validate that the circles are blue, as shown in Figure 7-4. 


B 
e 


ck 


Figure 7-4. Deployed application 
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To remove the application, use the CLI tool or the UI: 


argocd app delete bgd-app 


Discussion 


Although Argo CD deploys applications automatically, it uses some default conserva- 
tive strategies for safety reasons. 


Two of these are the pruning of deleted resources and the self-healing of the applica- 
tion in case a change was made in the Kubernetes cluster directly instead of through 
Git. 


By default, Argo CD will not delete (prune) any resource when it detects that it is no 
longer available in Git, and it will be in an OutOfSync status. If you want Argo CD to 
delete these resources, you can do it in two ways. 


The first way is by manually invoking a sync with the -prune option: 
argocd app sync --prune 


The second way is letting Argo CD delete pruned resources automatically by setting 
the prune attribute to true in the automated section: 
syncPolicy: 


automated: 
prune: true (1) 


@ Enables automatic pruning 


Another important concept affecting how the application is automatically updated is 
self-healing. 


Argo CD is configured not to correct any drift made manually in the cluster. For 
example, Argo CD will let the execution of a kubectl patch directly in the cluster 
change any configuration parameter of the application. 


Let’s see it in action. 
The color of the circle is set as an environment variable (COLOR). 


Now, let’s change the COLOR environment variable to green using the kubectl patch 
command. 


Run the following command in the terminal: 
kubectl -n bgd patch deploy/bgd \ 
--type='json' -p='[{"op": "replace", "path": "/ 
spec/template/spec/containers/0/env/0/value", "value":"green"}]' 


Wait for the rollout to happen: 


kubectl rollout status deploy/bgd -n bgd 
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If you refresh the browser, you should see green circles now, as shown in Figure 7-5. 


© 


@ 


Figure 7-5. Deployed application 


Looking over the Argo CD sync status, you'll see that it’s OutOfSync as the application 
and the definition in the Git repository (COLOR: blue) diverges. 


Argo CD will not roll back to correct this drift as the selfHeal property default is set 
to false. 


Let's remove the application and deploy a new one, but set selfHeal to true in this 
case: 


argocd app delete bgd-app 
Let’s enable the selfHealing property, and repeat the experiment: 


apiVersion: argoproj.io/vialpha1 
kind: Application 
metadata: 
name: bgd-app 
Namespace: argocd 
spec: 
destination: 
Namespace: bgd 
server: https://kubernetes.defauLlt.svc 
project: default 
source: 
path: ch07/bgd 
repoURL: https://github.com/gitops-cookbook/gitops-cookbook-sc.git 
targetRevision: main 
syncPolicy: 
automated: 
prune: true 
selfHeal: true @ 


@ selfHeal set to true to correct any drift 
And in the terminal apply the resource: 


kubectl apply -f bgd/heal-bgd-app.yaml 
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Repeat the previous steps: 


1. Open the browser to check that the circles are blue. 
2. Reexecute the kubectl -n bgd patch deploy/bgd ... command. 
3. Refresh the browser and check that the circles are still blue. 


Argo CD corrects the drift introduced by the patch command, synchronizing the 
application to the correct state defined in the Git repository. 


To remove the application, use the CLI tool or the UI: 


argocd app delete bgd-app 


See Also 


e Argo CD Automated Sync Policy 
e Argo CD Sync Options 


7.3 Kustomize Integration 


Problem 


You want to use Argo CD to deploy Kustomize manifests. 


Solution 
Argo CD supports several different ways in which Kubernetes manifests can be 
defined: 
e Kustomize 
e Helm 
e Ksonnet 
e Jsonnet 
You can also extend the supported ways to custom ones, but this is out of the scope of 


this book. 


Argo CD detects a Kustomize project if there are any of the following files: kustomiza- 
tion.yaml, kustomization.yml, or Kustomization. 


Lets deploy the same BGD application, but in this case, deployed as Kustomize 
manifests. 


Moreover, we'll set kustomize to override the COLOR environment variable to yellow. 
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The Kustomize file defined in the repository looks like this: 


apiVersion: kustomize.config.k8s.io/vibetal 
kind: Kustomization 
Namespace: bgdk 
resources: 
- ../base (13 
- bgdk-ns.yaml (2) 
patchesJson6902: © 
- target: O 
version: v1 
group: apps 
kind: Deployment 
name: bgd 
namespace: bgdk 
patch: |- (5 
- op: replace 


path: /spec/template/spec/containers/0/env/0/value 


value: yellow 


Specific file for creating a namespace 
Patches standard deployment files 


Patches the deployment file 


© © © 8 8 


Directory with standard deployment files (blue circles) 


Overrides the environment variable value to yellow 


You don’t need to create this file as it’s already stored in the Git 


repository. 


Create the following Application file to deploy the application: 


apiVersion: argoproj.io/vialpha1 
kind: Application 
metadata: 
name: bgdk-app 
Namespace: argocd 
spec: 
destination: 
Namespace: bgdk 
server: https://kubernetes.defauLlt.svc 
project: default 
source: 
path: ch07/bgdk/bgdk 


repoURL: https://github.com/gitops-cookbook/gitops-cookbook-sc.git 
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targetRevision: main 
syncPolicy: 
automated: {} 


At this point, we can apply the Application file to a running cluster by running the 
following command: 


kubectl apply -f bgdk/bgdk-app.yaml 
Access the service and you'll notice the circles are yellow instead of blue. 
To remove the application, use the CLI tool or the UI: 


argocd app delete bgdk-app 


Discussion 


We can explicitly specify which tool to use, overriding the default algorithm used by 
Argo CD in the Application file. For example, we can use a plain directory strategy 
regarding the presence of the kustomization.yaml file: 

source: 


directory: (1) 
recurse: true 


@ Overrides always use a plain directory strategy 


Possible strategies are: directory, chart, helm, kustomize, path, and plugin. 


Everything weve seen about Kustomize is valid when using Argo 
CD. 


See Also 


e Chapter 4 
e argo-cd/application-crd.yaml on GitHub 
e Argo CD Tool Detection 


7.4 Helm Integration 


Problem 


You want to use Argo CD to deploy Helm manifests. 
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Solution 


Argo CD supports installing Helm Charts to the cluster when it detects a Helm 


project in the deployment directory (when the Chart. yaml file is present). 


Let’s deploy the same BGD application, but in this case, deployed as a Helm manifest. 


The layout of the project is a simple Helm layout already created in the GitHub 


repository you've cloned previously: 


— Chart. yaml 
| charts 

ļ— templates 
| H NOTES. txt 

| }— _helpers.tpl 

| ļ— deployment. yaml 

| H| ns.yaml 

| ļ— service. yaml 

| ļ|— serviceaccount. yaml 

| L— tests 

| L— test-connection. yaml 
L— values.yaml 


Create a bgdh/bgdh-app.yaml file to define the Argo CD application: 


apiVersion: argoproj.io/vialpha1 
kind: Application 
metadata: 
name: bgdh-app 
Namespace: argocd 
spec: 
destination: 
Namespace: bgdh 
server: https://kubernetes.defauLlt.svc 
project: default 
source: 
path: ch07/bgdh 
repoURL: https://github.com/gitops-cookbook/gitops-cookbook-sc.git 
targetRevision: main 
syncPolicy: 
automated: {} 


At this point, we can apply the Application file into a running cluster by running the 


following command: 
kubectl apply -f bgdh/bgdh-app.yaml 
Validate the pod is running in the bgdh namespace: 


kubectl get pods -n bgdh 


NAME READY STATUS RESTARTS AGE 
bgdh-app-556c46fcd6-ctfkf 1/1 Running 0 5m43s 
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To remove the application, use the CLI tool or the UI: 


argocd app delete bgdh-app 


Discussion 


Argo CD populates build environment variables to Helm manifests (actually also 
Kustomize, Jsonnet, and custom tools support too). 


The following variables are set: 


e ARGOCD_APP_NAME 
e ARGOCD_APP_NAMESPACE 

e ARGOCD_APP_REVISION 

e ARGOCD_APP_SOURCE_PATH 

e ARGOCD_APP_SOURCE_REPO_URL 

e ARGOCD_APP_SOURCE_TARGET_REVISION 
e KUBE_VERSION 

e KUBE_API_VERSIONS 


In the following snippet, you can see the usage of the application name: 


apiVersion: argoproj.io/vialpha1 
kind: Application 
metadata: 
name: bgdh-app 
Namespace: openshift-gitops 
spec: 
destination: 
source: 
path: ch07/bgd 
helm: © 
parameters: @ 
- name: app © 
value: SARGOCD_APP_NAME @ 


Specific Helm section 
Extra parameters to set, same as setting them in values.yaml, but high preference 


The name of the parameter 


o © 8 8 


The value of the parameter, in this case from a Build Env var 
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Argo CD can use a different values.yaml file or set parameter values to override the 
ones defined in values.yami: 


argocd app set bgdh-app --values new-values.yaml 


argocd app set bgdh-app -p service. type=LoadBalancer 


Note that values files must be in the same Git repository as the Helm Chart. 


Argo CD supports Helm hooks too. 


See Also 


e Chapter 5 
e argo-cd/application-crd.yaml on GitHub 


7.5 Image Updater 


Problem 


You want Argo CD to automatically deploy a container image when it’s published. 


Solution 


Use Argo CD Image Updater to detect a change on the container registry and update 
the deployment files. 


One of the most repetitive tasks during development is deploying a new version of a 
container image. 


With a pure Argo CD solution, after the container image is published to a container 
registry, we need to update the Kubernetes/Kustomize/Helm manifest files pointing 
to the new container image and push the result to the Git repository. 


This process implies: 
1. Clone the repo 


2. Parse the YAML files and update them accordingly 
3. Commit and Push the changes 


These boilerplate tasks should be defined for each repository during the continuous 
integration phase. Although this approach works, it could be automated so the cluster 
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could detect a new image pushed to the container registry and update the current 
deployment file pointing to the newer version. 


This is exactly what Argo CD Image Updater (ArgoCD IU) does. It's a Kubernetes 
controller monitoring for a new container version and updating the manifests 
defined in the Argo CD Application file. 


The Argo CD IU lifecycle and its relationship with Argo CD are shown in Figure 7-6. 


Argo CD Image Image Cl 


Updater ra Registry 
2 L, = / os 


5. Monitor and find a Qe 4. At some point, a new 


6. Updates manifests with 
new container image | container image is pushed 
< 


new container image -., 


peceensnaaes 3. Kubernetes pulls the container 


a 
Config Git Reposito —— +> Kubernetes 
ceson  — È e 


Argo CD 


1. A change is detected a "h, 2. Argo CD applies the manifests 
7. The new change is detected 
Detect 
Deploy 
Take K 
action 


Figure 7-6. Argo CD Image Updater lifecycle 


At this time, Argo CD IU only updates manifests of Kustomize or Helm. In the 
case of Helm, it needs to support specifying the images tag using a parameter 
(image. tag). 


Let’s install the controller in the same namespace as Argo CD: 


kubectl apply -f \ 
https://raw.githubusercontent.com/argoproj-labs/argocd-imageupdater/v0.12.0/mani- 
fests/install.yaml -n argocd 


Validate the installation process, checking that the pod status of the controller is 
Running: 


kubectl get pods -n argocd 


NAME READY STATUS 
RESTARTS AGE 

argocd-image-updater -59c45cbc5c-kjjtp 1/1 Running 
0 40h 
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Before using Argo CD IU, we create a Kubernetes Secret representing the Git creden- 
tials, so the updated manifests can be pushed to the repository. The secret must be at 
the Argo CD namespace and, in this case, we name it git-creds. 


kubectl -n argocd create secret generic git-creds \ --from-literal=user 
name=<git_user> \ 
--from-literal=password=<git_password_or_token> 


Finally, let’s annotate the Application manifest with some special annotations so the 
controller can start monitoring the registry: 


image-list 
Specify one or more images (comma-separated-value) considered for updates. 


write-back-method 
Methods to propagate new versions. There are git and argocd methods imple- 
mented to update to a newer image. The Git method commits the change to 
the Git repository. Argo CD uses the Kubernetes/ArgoCD API to update the 
resource. 


There are more configuration options, but the previous ones are the most important 
to get started. 


Let's create an Argo CD Application manifest annotated with Argo CD IU 
annotations: 


apiVersion: argoproj.io/vialpha1 
kind: Application 
metadata: 
name: bgdk-app 
Namespace: argocd 
annotations: 
Ə argocd-image-updater.argoproj.io/image-list: myalias=quay.io/rhdevelopers/bgd 
argocd-image-updater.argoproj.io/write-back-method: git:secret:openshift- 
gitops/git-creds © 
argocd-image-updater.argoproj.io/git-branch: main (4) 
spec: 
destination: 
namespace: bgdk 
server: https://kubernetes.defauLlt.svc 
project: default 
source: 
path: ch07/bgdui/bgdk 
repoURL: https://github.com/gitops-cookbook/gitops-cookbook-sc.git 
targetRevision: main 
syncPolicy: 
automated: {} 


@ Adds annotations section 
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Sets the monitored image name 


Configures to use Git as write-back-method, setting the location of the creden- 
tials (<namespace>/<secretname>) 


© Sets the branch to push changes 


Now apply the manifest to deploy the application’s first version and enable Argo CD 
IU to update the repository when a new image is pushed to the container registry: 
kubectl apply -f bgdui/bgdui-app.yaml 


At this point, version 1.0.0 is up and running in the bgdk namespace, and you may 
access it as weve done before. Lets generate a new container version to validate that 
the new image is in the repository. 


To simplify the process, we'll tag the container with version 1.1.0 as it was a new one. 


Go to the Quay repository created at the beginning of this chapter, go to the tags 
section, push the gear icon, and select Add New Tag to create a new container, as 
shown in Figure 7-7. 


B rhdevelopers/bgd xy 


fi} Repository Tags 


Repository name Gear icon — 
® 4—— Tag section f i ors a 


9 100 © Ohoh- tamabie 1323M 
3 


latest 


© Wovign-vanable 2775 NB 


Figure 7-7. Tag container 


Set the tag to 1.1.0 value as shown in the figure Figure 7-8. 


Add Tag to Manifest sha256:e0187fae04d9 


one 


Figure 7-8. Tag container 
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After this step, you should have a new container created as shown in Figure 7-9. 


Wait for around two minutes until the change is detected and the controller triggers 
the repo update. 


€ Repositories ® rhdevelopers / bgd 
@ Repository Tags = 
» z | 1-30f3 | | Filter Tags... | 
TAG LAST MODIFIED } SECURITY SCAN SIZE EXPIRES MANIFEST 
D) 11.0 afewsecondsago C) 10 High - 18 fixable 132.3MB Never SHAZEE  cois7éacesds j + 2 
3 
1.0.0 2 days ago © 10 High - 18 fixable 132.3MB Never SHA256 co187Facoad9 i Q 
= latest 4 days ago © 10 Hioh - 14 fixable 277.5MB Never SAREE 7a95efasabos + & 
= 


Figure 7-9. Final result 


To validate the triggering process check the logs of the controller: 


kubectl logs argocd-image-updater -59c45cbc5c-kjjtp -f -n argocd 


time="2022-06-20T21:19:05Z" Level=info msg="Setting new image to quay.io/rhdevel 
opers/bgd:1.1.0" alias=myalias application=bgdk-app image_name=rhdeve Lopers/bgd 
image_tag=1.0.0 registry=quay.io 

time="2022-06-20T21:19:05Z" lLevel=info msg="Successfully updated image 'quay.io/ 
rhdevelopers/bgd:1.0.0' to 'quay.io/rhdevelopers/bgd:1.1.0', but pending spec 
update (dry run=false)" alias=myalias application=bgdk-app image_name=rhdevelop 
ers/bgd image_tag=1.0.0 registry=quay.io 

time="2022-06-20T21:19:05Z" Level=info msg="Committing 1 parameter update(s) for 
application bgdk-app" application=bgdk-app 


@ Detects the change and updates the image 


After that, if you inspect the repository, you'll see a new Kustomize file 
named .argocd-source-bgdk-app.yaml, updating the image value to the new con- 
tainer, as shown in Figure 7-10. 
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P main ~ | gitops-cookbook-sc / ch07 / bgdui / bgdk / argocd-source-bgdk-app.yaml Go to file 


® argocd-image-updater build: automatic update of bgdk-app -= Latest commit £36ba98 6 minutes ago ©) History 


A 0 contributors 


o 


3 lines (3 sloc) | 56 Bytes Raw | Blame | 2 ~- 0Ü 


1 kustomize: 
2 images: 
3 - quay.io/rhdevelopers/bgd:1.1.0 


Figure 7-10. New Kustomize file updating to the new container 


Now Argo CD can detect the change and update the cluster properly with the new 
image. 


To remove the application, use the CLI tool or the UI: 


argocd app delete bgdk-app 


Discussion 


An update strategy defines how Argo CD IU will find new versions. With no change, 
Argo CD IU uses a semantic version to detect the latest version. 


An optional version constraint field may be added to restrict which versions are 
allowed to be automatically updated. To only update patch versions, we can change 
the image- list annotation as shown in the following snippet: 


argocd-image-updater.argoproj.io/image-list: myalias=quay.io/rhdevelopers/bgd:1.2.x 
Argo CD Image Updater can update to the image that has the most recent build date: 
argocd-image-updater.argoproj.io/myalias.update-strategy: latest 
argocd-image-updater.argoproj.io/myimage.allow-tags: regexp:*[0-9a-f]{7}$ 1] 
@ Restricts the tags considered for the update 


The digest update strategy will use image digests to update your applications’ image 
tags: 


argocd-image-updater .argoproj.io/myalias.update-strategy: digest 


So far, the container was stored in a public registry. If the repository is private, Argo 
CD Image Updater needs read access to the repo to detect any change. 


First of all, create a new secret representing the container registry credentials: 


kubectl create -n argocd secret docker-registry quayio --docker-server=quay.io -- 
docker -username=SQUAY_USERNAME - -docker -password=$QUAY_PASSWORD 
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Argo CD Image Updater uses a ConfigMap as a configuration source, which is the 
place to register the private container registry. Create a new ConfigMap manifest 
setting the supported registries: 


apiVersion: v1 
kind: ConfigMap 
metadata: 
name: argocd-image-updater-config 1] 
data: 
registries.conf: | 
registries: (2) 
- name: RedHat Quay © 
api_url: https://quay.io (4) 
prefix: quay.io 
insecure: yes 
credentials: pullsecret:argocd/quayio (6 


Name of the Argo CD IU ConfigMap 
Place to register all registries 
A name to identify it 


URL of the service 


The prefix used in the container images 


O © © 86 8 8 


Gets the credentials from the quayio secret stored at argocd namespace 
Argo CD Image Updater commits the update with a default message: 


commit 3cafOaf8b7a26de70a641c696446bbe1cd04cea8 (HEAD -> main, origin/main) 
Author: argocd-image-updater <noreply@argoproj.io> 
Date: Thu Jun 23 09:41:00 2022 +0000 


build: automatic update of bgdk-app 


updates image rhdevelopers/bgd tag '1.0.0' to '1.1.0' 


We can update the default commit message to one that fits your requirements. 
Configure the git.commit-message-template key in ArgoCD IU argocd-image- 
updater -config ConfigMap with the message: 


apiVersion: v1 
kind: ConfigMap 
metadata: 
name: argocd-image-updater-config (1) 
data: 
git.user: alex (2) 
git.email: alex@example.com © 
git.commit-message-template: | (4) 
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build: automatic update of {{ .AppName }} (5) 


{{ range .AppChanges -}} (6) 
updates image {{ .Image }} tag '{{ .OldTag }}' to '{{ .NewTag }}' 0o00 
{{ end -}} 


Argo CD IU ConfigMap 

Commit user 

Commmit email 

Golang text/template content 

The name of the application 

List of changes performed by the update 
Image name 


Previous container tag 


060686 O 6 6 8 8 


New container tag 


Remember to restart the Argo CD UI controller when the Config 
Map is changed: 


kubectl rollout restart deployment argocd-image-updater 
-n argocd 


See Also 


e Argo CD Image Updater 


7.6 Deploy from a Private Git Repository 


Problem 
You want Argo CD to deploy manifests. 


Solution 


Use Argo CD CLI/UI or YAML files to register the repositories’ credential informa- 
tion (username/password/token/key). 
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In Argo CD, you have two ways to register a Git repository with its credentials. One 
way is using the Argo CD CLI/Argo CD UI tooling. To register a private repository in 
Argo CD, set the username and password by running the following command: 

argocd repo add https://github.com/argoproj/argocd-example-apps \ 

--username <username> --password <password> 
Alternatively, we can use the Argo CD UI to register it too. Open Argo CD UI ina 
browser, and click the Settings/Repositories button (the one with gears) as shown in 
Figure 7-11. 


Settings 


Repositories <-— Repositories > 
‘Configure connected repositories: 

Settings 
Certificates > 
Configure certificates for connecting Git repostiories 
GnuPG keys > 


Configure GnuPG public keys for commit verification 


Figure 7-11. Settings menu 


Then click the “Connect Repo using HTTPS” button and fill the form with the 
required data as shown in Figure 7-12. 


CONNECT SAVE AS CREDENTIALS TEMPLATE CANCEL 


Connect rep&using HTTPS 


Type Registers the 
ait repository 


Project 
Repository URL 
Username (optional) 


Password (optional) 


TLS client certificate (optional) 


Figure 7-12. Configuration of repository 
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Finally, click the Connect button to test that it’s possible to establish a connection and 
add the repository into Argo CD. 


The other way is to create a Kubernetes Secret manifest file with that repository and 
credentials information: 


apiVersion: v1 
kind: Secret 
metadata: 
name: private-repo 
Namespace: argocd (1) 
labels: 
argocd.argoproj.io/secret-type: repository (2) 
stringData: 
type: git 
url: https://github.com/argoproj/private-repo © 
password: my-password (4) 
username: my-username @ 


Create a secret in the Argo CD namespace 


Sets secret type as repository 


URL of the repository to register 


© © 8 8 


Password to access 


© Username to access 
If you apply this file, it will have the same effect as the manual approach. 


At this point, every time we define a repoURL value in the Application resource 
with a repository URL registered for authentication, Argo CD will use the registered 
credentials to log in. 


Discussion 


In addition to setting credentials such as username and password for accessing a 
private Git repo, Argo CD also supports other methods such as tokens, TLS client 
certificates, SSH private keys, or GitHub App credentials. 


Let’s see some examples using Argo CD CLI or Kubernetes Secrets. 
To configure a TLS client certificate: 
argocd repo add https://repo.example.com/repo.git \ 


--tls-client-cert-path ~/mycert.crt \ 
--tls-client-cert-key-path ~/mycert.key 
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For SSH, you just need to set the location of the SSH private key: 


argocd repo add git@github.com:argoproj/argocd-example-apps.git \ 
--ssh-privatekey-path ~/.ssh/id_rsa 


Or using a Kubernetes Secret: 


apiVersion: vi 
kind: Secret 
metadata: 
Name: private-repo 
Namespace: argocd 
labels: 
argocd.argoproj.io/secret-type: repository 
stringData: 
type: git 
url: git@github.com:argoproj/my-private-repository 
sshPrivateKey: | (1) 


@ Sets the content of the SSH private key 


If you are using the GitHub App method, you need to set the App ID, the App 
Installation ID, and the private key: 


argocd repo add https://github.com/argoproj/argocd-example-apps.git --github-app- 
id 1 --github-app-installation-id 2 --github-app-private-key-path test.private- 
key. pem 


Or using the declarative approach: 


apiVersion: v1 
kind: Secret 
metadata: 
name: github-repo 
namespace: argocd 
labels: 
argocd.argoproj.io/secret-type: repository 
stringData: 
type: git 
repo: https://ghe.example.com/argoproj/my-private-repository 
githubAppID: 1 @ 
githubAppInstallationID: 2 
githubAppEnterpriseBaseUrl: https: //ghe.example.com/api/v3 (2) 
githubAppPrivateKeySecret: | 


@ Sets GitHub App parameters 


© Only valid if GitHub App Enterprise is used 
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For the access token, use the account name as the username and the token in the 
password field. 


Choosing which strategy to use will depend on your experience managing Kubernetes 
Secrets. Remember that a Secret in Kubernetes is not encrypted but encoded in 
Base64, so it is not secured by default. 


We recommend using only the declarative approach when you've got a good strategy 
for securing the secrets. 


We've not discussed the Sealed Secrets project yet (we'll do so in the 
following chapter), but when using Sealed Secrets, the labels will 
be removed to avoid the SealedSecret object having a template 
section that encodes all the fields you want the controller to put in 
the unsealed Secret: 


spec: 
template: 
metadata: 


labels: 
"argocd.argoproj.io/secret-type": repository 


7.7 Order Kubernetes Manifests 


Problem 
You want to use Argo CD to deploy. 


Solution 
Use sync waves and resource hooks to modify the default order of applying manifests. 
Argo CD applies the Kubernetes manifests (plain, Helm, Kustomize) in a particular 
order using the following logic: 
1. By kind 
a. Namespaces 
b. NetworkPolicy 
c. Limit Range 
d. ServiceAccount 
e. Secret 
f. ConfigMap 
g. StorageClass 
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h. PersistentVolumes 
. ClusterRole 
j. Role 


jio 


k. Service 

1. DaemonSet 
m. Pod 

n. ReplicaSet 
o. Deployment 
. StatefulSet 
. Job 


. Ingress 


=m FO SO 


2. In the same kind, then by name (alphabetical order) 


Argo CD has three phases when applying resources: the first phase is executed 
before applying the manifests (PreSync), the second phase is when the manifests are 
applied (Sync), and the third phase is executed after all manifests are applied and 
synchronized (PostSync). 


Figure 7-13 summarizes these phases. 


Passage of Time 


Phases 
a — ae 


job deploy service route 
- delete on success 
e 0 ż 
\ Hook "K Hook 
W 
e3 $ 


Figure 7-13. Hooks and sync waves 


Resource hooks are scripts executed at a given phase, or if the Sync phase failed, you 
could run some rollback operations. 
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Table 7-1 lists the available resource hooks. 


Table 7-1. Resource hooks 
Hook Description Use case 
PreSync Executes prior to the application of the manifests Database migrations 


Sync Executes at the same time as manifests Complex rolling update strategies like canary releases or 
dark launches 


PostSync Executes after all Sync hooks have completed and Run tests to validate deployment was correctly done 
were successful (healthy) 


SyncFail Executes when the sync operation fails Rollback operations in case of failure 


Skip Skip the application of the manifest When manual steps are required to deploy the 
application (i.e., releasing public traffic to new version) 


Hooks are defined as an annotation named argocd.argoproj.io/hook to a Kuber- 
netes resource. In the following snippet, a PostSync manifest is defined: 


apiVersion: batch/v1 
kind: Job 
metadata: 
name: todo-insert @ 
annotations: 
argocd.argoproj.io/hook: PostSync (2) 


@ Jobs name 


Sets when the manifest is applied 


Deletion Policies 


A hook is not deleted when finished; for example, if you run a Kubernetes Job, it'll 
remain Completed. 


This might be the desired state, but we can specify to automatically delete these 
resources if annotated with argocd.argoproj.io/hook-delete-policy and the pol- 
icy value is set. 


Supported policies are: 


Policy Description 


HookSucceeded Deleted after the hook succeeded 
HookFailed Deleted after the hook failed 


BeforeHookCreation Deleted before the new one is created 
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A sync wave is a way to order how Argo CD applies the manifests stored in Git. 


All manifests have zero waves by default, and the lower values go first. Use the 
argocd.argoproj.io/sync-wave annotation to set the wave number to a resource. 


For example, you might want to deploy a database first and then create the database 
schema; for this case, you should set a sync-wave lower in the database deployment 
file than in the job for creating the database schema, as shown in the following 
snippet: 


apiVersion: apps/v1 
kind: Deployment 
metadata: 
name: postgresql (1) 
namespace: todo 
annotations: 
argocd.argoproj.io/sync-wave: "0" (2) 


apiVersion: batch/v1 
kind: Job 
metadata: 
name: todo-table @ 
Namespace: todo 
annotations: 
argocd.argoproj.io/sync-wave: "1" (4) 


PostgreSQL deployment 
Sync wave for PostgreSQL deployment is 0 


Name of the Job 


o © 8 8 


Job executed when PostgreSQL is healthy 


Discussion 
When Argo CD starts applying the manifests, it orders the resources in the following 


way: 


1. Phase 
2. Wave (lower precedence first) 
3. Kind 
4. Name 


Let's deploy a more significant application with deployment files, sync waves, and 
hooks. 
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The sample application deployed is a TODO application connected with a database 
(PostgreSQL) to store TODOs. To deploy the application, some particular order 
needs to be applied; for example, the database server must be running before creating 
the database schema. Also, when the whole application is deployed, we insert some 
default TODOs into the database to run a post-sync manifest. 


The overall process is shown in Figure 7-14. 


u 
a 
f=) 
z 


Wave -1: Creates todo Namespace 
Create Namespace 


— Wave 0: Deploys PostgreSQL 
Deploy PostgreSQL 


JSON 


— Wave 1: Updates PostgreSQL 
Create DB tables 


— Wave 2: Deploys Todo App 
Deploy Todo App 


au 
a 
[=] 
z 


= Wave 3: Configures Routes to access 
Configure Route 


— PostSync: Inserts Todo 
Make HTTP call 


Figure 7-14. Todo app 
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Create an Application resource pointing out to the application: 


apiVersion: argoproj.io/vialpha1 
kind: Application 
metadata: 
Name: todo-app 
Namespace: argocd 
spec: 
destination: 
Namespace: todo 
server: https://kubernetes.defauLt.svc 
project: default 
source: 
path: ch07/todo 
repoURL: https://github.com/gitops-cookbook/gitops-cookbook-sc.git 
targetRevision: main 
syncPolicy: 
automated: 
prune: true 
selfHeal: false 
syncOptions: 
- CreateNamespace=true 


In the terminal, apply the resource, and Argo CD will deploy all applications in the 
specified order. 


See Also 


e gitops-engine/sync_tasks.go on GitHub 
7.8 Define Synchronization Windows 


Problem 


You want Argo CD to block or allow application synchronization depending on time. 


Solution 


Argo CD has the sync windows concept to configure time windows where application 
synchronizations (applying new resources that have been pushed to the repository) 
will either be blocked or allowed. 


To define a sync window, create an AppProject manifest setting the kind (either 
allow or deny), a schedule in cron format to define the initial time, a duration 
of the window, and which resources the sync window is applied to (Application, 
namespaces, or clusters). 
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About Cron Expressions 


A cron expression represents a time. It’s composed of the following fields: 


r~~ minute (0 - 59) 

m~~ hour (0 - 23) 

r~~ day of the month (1 - 31) 
| m month (1 - 12) 

| 

* 


| m day of the week (0 - 6) 
k k 


o 


The AppProject resource is responsible for defining these windows where synchroni- 
zations are permitted/blocked. 


Create a new file to permit synchronizations only from 22:00 to 23:00 (just one hour) 
and for Argo CD Applications whose names end in -prod: 


© © 6 Ọ 8 8 


apiVersion: argoproj.io/vialpha1 
kind: AppProject 
metadata: 
name: default 
spec: 

syncWindows: (1) 

- kind: allow @ 
schedule: '0 22 * * *' © 
duration: 1h @ 
applications: (5) 

- '*-prod' (6) 


List of windows 

Allow syncs 

Only at 22:00 

For 1 hour (23:00) 

Sets the applications that affect this window 


Regular expression matching any application whose name ends with -prod 


Discussion 


We cannot perform a sync of the application (neither automatic nor manual) when 
it’s not the time configured in the time window defined in the AppProject manifest. 
However, we can configure a window to allow manual syncs. 
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Using the CLI tool: 


argocd proj windows enable-manual-sync <PROJECT ID> 


Also, manual sync can be set in the YAML file. In the following example, we're setting 


manual synchronization for the namespace default, denying synchronizations at 22: 
for one hour and allowing synchronizations in prod-cluster at 23:00 for one hour: 


apiVersion: argoproj.io/vialpha1 
kind: AppProject 
metadata: 
name: default 
Namespace: argocd 
spec: 

syncWindows: 

- kind: deny 1] 
schedule: '0 22 * * *' 
duration: 1h 
ManualSync: true (2) 
namespaces: © 
- bgd 

- kind: allow 
schedule: '0 23 * * *' 
duration: 1h 
clusters: @ 

- prod-cluster 


Block synchronizations 
Enable manual sync to default namespace 


Configure namespaces to block 


o © 8 8 


Configure clusters to allow syncs at 23:00 


00 


We can inspect the current windows from the UI by going to the Settings > Projects 


—> default > windows tab or by using the argocd CLI tool: 


argocd proj windows list default 


ID STATUS KIND SCHEDULE DURATION APPLICATIONS NAMESPACES CLUSTERS 


MANUALSYNC 

0 Inactive deny 022 %* * * th - bgd - 

Enabled 

1 Inactive allow 0 23 * * * íh - - prod-cluster 
Disabled 
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CHAPTER 8 
Advanced Topics 


In the previous chapter, you had an overview of implementing GitOps workflows 
using Argo CD recipes. Argo CD is a famous and influential open source project that 
helps with both simple use cases and more advanced ones. In this chapter, we will 
discuss topics needed when you move forward in your GitOps journey, and you need 
to manage security, automation, and advanced deployment models for multicluster 
scenarios. 


Security is a critical aspect of automation and DevOps. DevSecOps is a new definition 
of an approach where security is a shared responsibility throughout the entire IT 
lifecycle. Furthermore, the DevSecOps Manifesto specifies security as code to operate 
and contribute value with less friction. And this goes in the same direction as GitOps 
principles, where everything is declarative. 


On the other hand, this also poses the question of avoiding storing unencrypted 
plain-text credentials in Git. As stated in the book Path to GitOps by Christian 
Hernandez, Argo CD luckily currently provides two patterns to manage security in 
GitOps workflows: 


e Storing encrypted secrets in Git, such as with a Sealed Secret (see Recipe 8.1) 


e Storing secrets in external services or vaults, then storing only the reference to 
such secrets in Git (see Recipe 8.2) 


The chapter then moves to advanced deployment techniques, showing how to 
manage webhooks with Argo CD (see Recipe 8.3) and with ApplicationSets (see 
Recipe 8.4). ApplicationSets is a component of Argo CD that allows management 
deployments of many applications, repositories, or clusters from a single Kubernetes 
resource. In essence, a templating system for the GitOps application is ready to be 
deployed and synced in multiple Kubernetes clusters (see Recipe 8.5). 
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Last but not least, the book ends with a recipe on Progressive Delivery for Kuber- 
netes with Argo Rollouts (Recipe 8.6), useful for deploying the application using an 
advanced deployment technique such as blue-green or canary. 


8.1 Encrypt Sensitive Data (Sealed Secrets) 


Problem 


You want to manage Kubernetes Secrets and encrypted objects in Git. 


Solution 


Sealed Secrets is an open source project by Bitnami used to encrypt a Kubernetes 
Secrets into a SealedSecret Kubernetes Custom Resource, representing an encrypted 
object safe to store in Git. 


Sealed Secrets uses public-key cryptography and consists of two main components: 


e A Kubernetes controller that has knowledge about the private and public key 
used to decrypt and encrypt encrypted secrets and is responsible for reconcilia- 
tion. The controller also supports automatic secret rotation for the private key 
and key expiration management in order to enforce the re-encryption of secrets. 


e kubeseal, a CLI used by developers to encrypt their secrets before committing 
them to a Git repository. 


The SealedSecret object is encrypted and decrypted only by the SealedSecret 
controller running in the target Kubernetes cluster. This operation is exclusive only 
to this component, thus nobody else can decrypt the object. The kubeseal CLI 
allows the developer to take a normal Kubernetes Secret resource and convert it to a 
SealedSecret resource definition as shown in Figure 8-1. 


In your Kubernetes cluster with Argo CD, you can install the kubeseal CLI for your 
operating system from the GitHub project’s releases. At the time of writing this book, 
we are using version 0.18.2. 


On macOS, kubeseal is available through Homebrew as follows: 


brew install kubeseal 
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Developer 


Commits SealedSecret Gets encrypted secret 


SealedSecret 
with encrypted 


secret 
SealedSecret 


Operator 


Figure 8-1. Sealed Secrets with GitOps 


After you install the CLI, you can install the controller as follows: 


kubectl create \ 


-f https://github.com/bitnami-labs/sealed-secrets/releases/download/0.18.2/control- 
ler.yaml 


You should have output similar to the following: 


serviceaccount/sealed-secrets-controller created 
deployment.apps/sealed-secrets-controller created 
customresourcedefinition.apiextensions.k8s.io/sealedsecrets.bitnami.com created 
service/sealed-secrets-controller created 
rolebinding.rbac.authorization.k8s.io0/sealed-secrets-controller created 
rolebinding.rbac.authorization.k8s.io/sealed-secrets-service-proxier created 
role.rbac.authorization.k8s.io0/sealed-secrets-service-proxier created 
role.rbac.authorization.k8s.io/sealed-secrets-key-admin created 
clusterrolebinding.rbac.authorization.k8s.io/sealed-secrets-controller created 
clusterrole.rbac.authorization.k8s.io/secrets-unsealer created 


As an example, let’s create a Secret for the Pac-Man game deployed in Chapter 5: 


kubectl create secret generic pacman-secret \ 
--from-literal=user=pacman \ 
--from-literal=pass=pacman 


You should have the following output: 
secret/pacman-secret created 
And here you can see the YAML representation: 


kubectl get secret pacman-secret -o yaml 
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apiVersion: v1 
data: 
pass: cGFjbWFu 
user: cGFjbWFu 
kind: Secret 
metadata: 
name: pacman-secret 
Namespace: default 
type: Opaque 


Now, you can convert the Secret into a SealedSecret in this way: 


kubectl get secret pacman-secret -o yaml \ 
| kubeseal -o yaml > pacman-sealedsecret.yaml 


apiVersion: bitnami.com/vialpha1 
kind: SealedSecret 
metadata: 
creationTimestamp: null 
name: pacman-secret 
Namespace: default 
spec: 
encryptedData: 1] 
pass: AgBJR1AgZ5Gu5NOVSG1E8SKBcdB3QSDdzZka3RRYUWV7z8g7ccQOdGc1suVOP8wXx/ 
ZpPmIMp8+urPYG62k4EZRUjuu/Vg2E1nSbsGBh9eKu3Na06tGSF3eGk6PZN6XtRhDeER4uU7MG5pj/ 
+FXRAKcy8Z6Rf zbVEGG/QIQ4z0ecSNdImGO7ERMm1Q+LPNGvph2Svx8aCgFLqRsdLhFyvwb 
TyB3XnmFHrPr+2DynxeN8XVMoMkRYXgVc6GAoxUK7CnC3ELpuy7 LIdPwc5QBx9kUVFra83LX8/KxeaJ 
wyCqvscIGjtcxUtpTpF5jm1t1DSRRNbc4m+7pTwImnRiUuaMVeujaBco4521yTkhSiEPjnj 
vUt+VZKO1NVoeNunglazp15rFwTvmiQ5PAtbiUXpT 733zCr60QBgSxPg31vw98+u+RcIHvaMIoDCqaX 
xUdcn2JkUF+bZXtxNmIRTAiQVQ1VEPmr ZxpvZcUh/PPC4L/RFWrQWnOzKRyqLq9wRoSLPbKyvMxX 
naxHOv3USGIktmtJLGjlLXoW/i+HIoSeMFSOmMUAZOF 5M5gweOhtxKGh3Y74ZDn5PbVA/ 
9kbkuWgvPNGDZL924Dm6AyMSgoHECr /RRTm1e22K9Bf PASARZuUGA6paqb9h1XEqyqesZgMOR8PLiy 
Luu+tpqydROSiYLc5VLtdjzplyyy9Xmw6Aa3/4SB+4t SwXSUUrB5yc= 
user: AgBhYDZQzOwinetPceZL897aibTYp4QPGFvP6ZhDyuUAx 
OWXBQ7 jBA3KPUgLvP8vBcxLAcS7HpKcDSgCdi47D2WhShdBR4 jWJufwKkmR3j+ayTdw72t3ALpQhTYIOiMY 
TiNdRO/o3vf0jeNMt/oWCRsifqBxZalIShE53rAFEjEA6D7CuCDXu8BHk1DpSr 79d5Au4puzpH 
VODh+v1T+Yef3k7DUoSnbYEh3CvuRweiugs LY8G000b28 j38wdyxm3GIrexa+M/ 
ZId01hxZ6jz4edv6ejdZfmQNdru3c6Lml jWwc0+QUeOMqFi4ZF /YNUsiojI+781n1m3K/ 
gikKcyPLn@skD7DyeKPoukoN6W5P710uFSkF+VgleejDaxuA7bK3PEaUgv79KFC9aEEnBr / 
7op7HY7X6aMDahmLUc/+zDhf zQvwnC2wc j4B8M20BFa2ic2PmGzr IWhlLBbs10gnpehtG 
SETq+YRDHOa LWOdFBq1U8qn6QA8 Iw6ewu8GTe Le3z LPLaADi506Lr JbIZNLYO+PutWf js9ScVVEJy+1I9BGd 
yT6tiA/4v4cxH6ygG6NzWkqxSaYyNrWWXtLhOlqyCpTZ 
tUwHnF+0LB3gCpDZPx+NwTe2Kn0jYOc83LULh5PJO9OASWWqZaRQyE 
LeL6y6mVekQFWHGFK6t57Vb7Z3+5XICgQn+xFLkj3SIZOME5D4+DSsUDS1f yL8ul= 
template: 
data: null 
metadata: 
creationTimestamp: null 
name: pacman-secret 
Namespace: default 
type: Opaque 


@ Here you find the data encrypted by the Sealed Secrets controller. 
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Now you can safely push your SealedSecret to your Kubernetes manifests repo and 
create the Argo CD application. Here’s an example from this book’s repository: 

argocd app create pacman \ 

--repo https://github.com/gitops-cookbook/pacman-kikd-manifests.git \ 

--path 'k8s/sealedsecrets' \ 

--dest-server https://kubernetes.default.svc \ 


--dest-namespace default \ 
--sync-policy auto 


Check if the app is running and healthy: 
argocd app list 
You should get output similar to the following: 


NAME CLUSTER NAMESPACE PROJECT STATUS HEALTH & 
SYNCPOLICY CONDITIONS 

REPO PATH TARGET 
pacman https://kubernetes.default.svc default default Synced Healthy 
<none> <none> https: //github.com/gitops-cookbook/pacman-kikd- 
manifests.git k8s/sealedsecrets 


8.2 Encrypt Secrets with ArgoCD (ArgoCD + HashiCorp 
Vault + External Secret) 


Problem 


You want to avoid storing credentials in Git and you want to manage them in external 
services or vaults. 


Solution 


In Recipe 8.1 you saw how to manage encrypted data in Git following the GitOps 
declarative way, but how do you avoid storing even encrypted credentials with 
GitOps? 


One solution is External Secrets, an open source project initially created by GoDaddy, 
which aims at storing secrets in external services or vaults from different vendors, 
then storing only the reference to such secrets in Git. 


Today, External Secrets supports systems such as AWS Secrets Manager, HashiCorp 
Vault, Google Secrets Manager, Azure Key Vault, and more. The idea is to provide a 
user-friendly abstraction for the external API that stores and manages the lifecycles of 
the secrets. 


In depth, ExternalSecrets is a Kubernetes controller that reconciles Secrets into the 
cluster from a Custom Resource that includes a reference to a secret in an external 
key management system. The Custom Resource SecretStore specifies the backend 
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containing the confidential data, and how it should be transformed into a Secret by 
defining a template, as you can see in Figure 8-2. The SecretStore has the configura- 
tion to connect to the external secret manager. 


Thus, the ExternalSecrets objects can be safely stored in Git, as they do not contain 
any confidential information, but just the references to the external services manag- 
ing credentials. 


Developer 
Commits 
ExternalSecret with 
correct reference 
to secret in key 
management system GitOps 


operator 


ExternalSecret Premal oeeret 
with reference with reference ExternalSecret 
to a secret Deploys toa secret operator 


secret 


Figure 8-2. External Secrets with Argo CD 


You can install External Secrets with a Helm Chart as follows. At the time of writing 
this book, we are using version 0.5.9: 


helm repo add external-secrets https://charts.external-secrets.io 


helm install external-secrets \ 
external-secrets/external-secrets \ 
-n external-secrets \ 
--create-namespace 


You should get output similar to the following: 


NAME: external-secrets 

LAST DEPLOYED: Fri Sep 2 13:09:53 2022 
NAMESPACE: external-secrets 

STATUS: deployed 

REVISION: 1 

TEST SUITE: None 

NOTES: 

external-secrets has been deployed successfully! 
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In order to begin using ExternalSecrets, you will need to set up a SecretStore or 
ClusterSecretStore resource (for example, by creating a vault SecretStore). 


More information on the different types of SecretStores and how to configure them 
can be found in our GitHub page. 


You can also install the External Secrets Operator with OLM from 
OperatorHub.io. 


As an example with one of the providers supported, such as HashiCorp Vault, you 
can do the following. 


First download and install HashiCorp Vault for your operating system and get your 
Vault Token. Then create a Kubernetes Secret as follows: 


export VAULT_TOKEN=<YOUR_TOKEN> 

kubectl create secret generic vault-token \ 
--from-literal=token=$VAULT_TOKEN \ 
-n external-secrets 


Then create a SecretStore as a reference to this external system: 


apiVersion: external-secrets.io/vibetal 
kind: SecretStore 
metadata: 
name: vault-secretstore 
Namespace: default 
spec: 
provider: 
vault: 
server: "http://vault.local:8200" 1) 
path: "secret" 
version: "v2" 
auth: 
tokenSecretRef: 
name: "vault-token" @ 
key: "token" © 
namespace: external-secrets 


© Hostname where your Vault is running 
Name of the Kubernetes Secret containing the vault token 


Key to address the value in the Kubernetes Secret containing the vault token 
content: 


kubectl create -f vault-secretstore.yaml 
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Now you can create a Secret in your Vault as follows: 
vault kv put secret/pacman-secrets pass=pacman 
And then reference it from the ExternalSecret as follows: 


apiVersion: external-secrets.io/vibetal 
kind: ExternalSecret 
metadata: 
Name: pacman-externalsecrets 
Namespace: default 
spec: 
refreshInterval: "15s" 
secretStoreRef: 
name: vault-secretstore 
kind: SecretStore 
target: 
name: pacman-externalsecrets 
data: 
- secretKey: token 
remoteRef : 
key: secret/pacman-secrets 
property: pass 


kubectl create -f pacman-externalsecrets.yaml 


Now you can deploy the Pac-Man game with Argo CD using External Secrets as 
follows: 

argocd app create pacman \ 

--repo https://github.com/gitops-cookbook/pacman-kikd-manifests.git \ 

--path 'k8s/externalsecrets' \ 

--dest-server https://kubernetes.default.svc \ 


--dest-namespace default \ 
--sync-policy auto 


8.3 Trigger the Deployment of an Application 
Automatically (Argo CD Webhooks) 


Problem 


You don’t want to wait for Argo CD syncs and you want to immediately deploy an 
application when a change occurs in Git. 


Solution 


While Argo CD polls Git repositories every three minutes to detect changes to 
the monitored Kubernetes manifests, it also supports an event-driven approach 
with webhooks notifications from popular Git servers such as GitHub, GitLab, or 
Bitbucket. 
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Argo CD Webhooks are enabled in your Argo CD installation and available at the 
endpoint /api/webhooks. 


To test webhooks with Argo CD using Minikube you can use Helm to install a local 
Git server such as Gitea, an open source lightweight server written in Go, as follows: 


helm repo add gitea-charts https://dl.gitea.io/charts/ 
helm install gitea gitea-charts/gitea 


You should have output similar to the following: 


helm install gitea gitea-charts/gitea 

"gitea-charts" has been added to your repositories 
NAME: gitea 

LAST DEPLOYED: Fri Sep 2 15:04:04 2022 

NAMESPACE: default 

STATUS: deployed 

REVISION: 1 

NOTES: 

1. Get the application URL by running these commands: 


echo "Visit http://127.0.0.1:3000 to use your application" 
kubectl --namespace default port-forward svc/gitea-http 3000:3000 


Log in to the Gitea server with the default credentials you find the 
in the values.yaml file from the Helm Chart here or define new 
ones via overriding them. 


Import the Pac-Man manifests repo into Gitea. 


Configure the Argo app: 


argocd app create pacman-webhook \ 


repo http://gitea-http.default.svc:3000/gitea_admin/pacman-kikd-manifests.git \ 


-dest-server https://kubernetes.default.svc \ 
-dest-namespace default \ 

-path k8s \ 

-sync-policy auto 


To add a webhook to Gitea, navigate to the top-right corner and click Settings. Select 
the Webhooks tab and configure it as shown in Figure 8-3: 


e Payload URL: http://localhost:9090/api/webhooks 


e Content type: application/json 
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Add Webhook gy 


Integrate Gitea into your repository. 


Target URL* | 


https://localhost:9090/api/webhook 


HTTP Method 


POST Sd 


POST Content Type 


application/json X 


Secret 


Trigger On: 

@ Push Events 

© All Events 

© Custom Events... 


Branch filter 


* 


a — Tia — = : = ——— — | 
Branch whitelist for push, branch creation and branch deletion events, specified as glob pattern. If empty or *, events for all branches are reported. See 
github.com/gobwas/glob documentation for syntax. Examples: master, {master, release*}. 


(¥) Active 
Information about triggered events will be sent to this webhook URL. 


Figure 8-3. Gitea Webhooks 


You can omit the Secret for this example; however, it’s best practice 
to configure secrets for your webhooks. Read more from the docs. 


Save it and push your change to the repo on Gitea. You will see a new sync from Argo 
CD immediately after your push. 


8.4 Deploy to Multiple Clusters 


Problem 


You want to deploy an application to different clusters. 


Solution 


Argo CD supports the ApplicationSet resource to “templetarize” an Argo CD 
Application resource. It covers different use cases, but the most important are: 
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e Use a Kubernetes manifest to target multiple Kubernetes clusters. 


e Deploy multiple applications from one or multiple Git repositories. 


Since the ApplicationSet is a template file with placeholders to substitute at run- 
time, we need to feed these with some values. For this purpose, ApplicationSet has 
the concept of generators. 


A generator is responsible for generating the parameters, which will finally be 
replaced in the template placeholders to generate a valid Argo CD Application. 


Create the following ApplicationSet: 


apiVersion: argoproj.io/vialpha1 
kind: ApplicationSet 
metadata: 
name: bgd-app 
Namespace: argocd 
spec: 
generators: @ 
- list: 
elements: @ 
- cluster: staging 
url: https://kubernetes.default.svc 
location: default 
- cluster: prod 
url: https://kubernetes.default.svc 
location: app 
template: © 
metadata: 
name: '{{cluster}}-app' (4) 
spec: 
project: default 
source: 
repoURL: https://github.com/gitops-cookbook/gitops-cookbook-sc.git 
targetRevision: main 
path: ch08/bgd-gen/{{cluster}} 
destination: 
server: '{{url}}' (5) 
namespace: '{{location}}' 
syncPolicy: 
syncOptions: 
- CreateNamespace=true 


Defines a generator 
Sets the value of the parameters 


Defines the Application resource as a template 


© © 8 8 


cluster placeholder 
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© url placeholder 


Apply the previous file by running the following command: 


kubectl apply -f bgd-application-set.yaml 


When this ApplicationSet is applied to the cluster, Argo CD generates and automat- 


ically registers two Application resources. The first one is: 


apiVersion: argoproj.io/vialpha1 
kind: Application 
metadata: 
name: staging-app 
spec: 
project: default 
source: 
path: ch08/bgd-gen/staging 
repoURL: https://github.com/example/app.git 
targetRevision: HEAD 
destination: 
Namespace: default 
server: https://kubernetes.defauLlt.svc 


And the second one: 


apiVersion: argoproj.io/vialpha1 
kind: Application 
metadata: 
Name: prod-app 
spec: 
project: default 
source: 
path: ch08/bgd-gen/prod 
repoURL: https://github.com/example/app.git 
targetRevision: HEAD 
destination: 
Namespace: app 
server: https://kubernetes.defauLlt.svc 


Inspect the creation of both Application resources by running the following 


command: 


# Remember to login first 


argocd login --insecure --grpc-web $argoURL --username admin --password SargoPass 


argocd app list 
And the output should be similar to (trunked): 


NAME CLUSTER NAMESPACE 
prod-app https://kubernetes.default.svc app 
staging-app https://kubernetes.default.svc default 
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Delete both applications by deleting the ApplicationSet file: 


kubectl delete -f bgd-application-set.yaml 


Discussion 


We've seen the simplest generator, but there are eight generators in total at the time of 
writing this book: 


List 
Generates Application definitions through a fixed list of clusters. (Its the one 
we've seen previously). 


Cluster 
Similar to List but based on the list of clusters defined in Argo CD. 

Git 
Generates Application definitions based on a JSON/YAML properties file within 
a Git repository or based on the directory layout of the repository. 


SCM Provider 
Generates Application definitions from repositories within an organization. 


Pull Request 
Generates Application definitions from open pull requests. 


Cluster Decision Resource 
Generates Application definitions using duck-typing. 


Matrix 
Combines values of two separate generators. 


Merge 
Merges values from two or more generators. 


In the previous example, we created the Application objects from a fixed list of 
elements. This is fine when the number of configurable environments is small; in 
the example, two clusters refer to two Git folders (ch08/bgd-gen/staging and ch08/ 
bgd-gen/prod). In the case of multiple environments (which means various folders), 
we can dynamically use the Git generator to generate one Application per directory. 


Let’s migrate the previous example to use the Git generator. As a reminder, the Git 
directory layout used was: 


bgd-gen 
|— staging 
| H| ...yaml 
L— prod 
H| ...yaml 
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Create a new file of type ApplicationSet generating an Application for each direc- 
tory of the configured Git repo: 


apiVersion: argoproj.io/vialpha1 
kind: ApplicationSet 
metadata: 
name: cluster-addons 
Namespace: openshift-gitops 
spec: 
generators: 
- git: (1) 
repoURL: https://github.com/gitops-cookbook/gitops-cookbook-sc.git 
revision: main 


directories: 
- path: ch08/bgd-gen/* (2) 
template: 

metadata: 

name: '{{path[0]}}{{path[2]}}' @ 
spec: 

project: default 

source: 


repoURL: https://github.com/gitops-cookbook/gitops-cookbook-sc.git 
targetRevision: main 
path: '{{path}}' © 
destination: 
server: https://kubernetes.default.svc 
Namespace: '{{path.basename}}' Q 


Configures the Git repository to read layout 
Initial path to start scanning directories 


Application definition 


© © 8 8 


The directory paths within the Git repository matching the path wildcard (stag 
ing or prod) 


© Directory path (full path) 


© The rightmost pathname 
Apply the resource: 
kubectl apply -f bgd-git-application-set.yaml 
Argo CD creates two applications as there are two directories: 


argocd app list 


NAME CLUSTER NAMESPACE 
ch08prod https://kubernetes.default.svc prod 
ch08staging https://kubernetes.default.svc staging 
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Also, this generator is handy when your application is composed of different compo- 
nents (service, database, distributed cache, email server, etc.), and deployment files 
for each element are placed in other directories. Or, for example, a repository with all 
operators required to be installed in the cluster: 


app 
ļ— tekton-operator 


| H| ...yaml 


|— prometheus-operator 


| H| ...yaml 
L— istio-operator 


H| ...yaml 


Instead of reacting to directories, Git generator can create Application objects with 
parameters specified in JSON/YAML files. 


The following snippet shows an example JSON file: 


{ 
"cluster": { 
"name": "staging", 
"address": "https://1.2.3.4" 
} 
} 


This is an excerpt of the ApplicationSet to react to these files: 


apiVersion: argoproj.io/vialpha1 
kind: ApplicationSet 
metadata: 
Name: guestbook 
spec: 
generators: 
- git: 
repoURL: https://github.com/example/app.git 
revision: HEAD 
files: 
- path: "app/**/config. json" (1) 
template: 
metadata: 
name: '{{cluster.name}}-app' (2) 


@ Finds all config.json files placed in all subdirectories of the app 


@ Injects the value set in config.json 


This ApplicationSet will generate one Application for each config.json file in the 
folders matching the path expression. 
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See Also 


e Argo CD Generators 
e Duck Types 


8.5 Deploy a Pull Request to a Cluster 


Problem 


You want to deploy a preview of the application when a pull request is created. 


Solution 


Use the pull request generator to automatically discover open pull requests within a 
repository and create an Application object. 


Lets create an ApplicationSet reacting to any GitHub pull request annotated with 
the preview label created on the configured repository. 


Create a new file named bgd-pr-application-set.yaml with the following content: 


apiVersion: argoproj.io/vialpha1 
kind: ApplicationSet 
metadata: 
name: myapps 
Namespace: openshift-gitops 
spec: 
generators: 
- pullRequest: 
github: (1) 
owner: gitops-cookbook (2) 
repo: gitops-cookbook-sc © 
labels: O 
- preview 
requeueAfterSeconds: 60 (5) 
template: 
metadata: 
name: 'myapp-{{branch}}-{{number}}' (6) 
spec: 
source: 
repoURL: 'https://github.com/gitops-cookbook/gitops-cookbook-sc.git' 
targetRevision: '{{head_sha}}' (7) 
path: ch08/bgd-pr 
project: default 
destination: 
server: https://kubernetes.default.svc 
namespace: '{{branch}}-{{number}}' 


@ GitHub pull request generator 
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Organization/user 

Repository 

Select the target PRs 

Polling time in seconds to check if there is a new PR (60 seconds) 


Sets the name with branch name and number 


© O O © 8 8 


Sets the Git SHA number 
Apply the previous file by running the following command: 
kubectl apply -f bgd-pr-application-set.yaml 


Now, if you list the Argo CD applications, you'll see that none are registered. The 
reason is there is no pull request yet in the repository labeled with preview: 


argocd app list 
NAME CLUSTER NAMESPACE PROJECT STATUS 


Create a pull request against the repository and label it with preview. 


In GitHub, the pull request window should be similar to Figure 8-4. 


Update bgd-deployment.yaml #1 tait || <> code ~ 
lordofthejars wants to merge 1 commit into main from lordofthejars-patch-1 (F 
| &) Conversation 0 | -œ Commits 1 E} Checks © Files changed 1 +2 -2 MEMET 
Q 4 lordofthejars commented 25 seconds ago O Reviewers B 
No reviews 
No description provided. a 
Still in progress? Convert to draft 
> @ Update bgd-deptoyment. yam Verified) 78af6f5 
Assignees fr} 


No one—assign yourself 
© @lordofthejars added the (D label now 


Labels & 
Add more commits by pushing to the Lordofthejars-patch-1 branch on gitops-cookbook/gitops-cookbook-sc. 
(~) This branch has no conflicts with the base branch Projects & 
Merging can be performed automatically. köns yer 
ME E. MAE You can also open this in GitHub Desktop or view command line instructions. Milestone & 


No milestone 


Figure 8-4. Pull request in GitHub 
Wait for one minute until the ApplicationSet detects the change and creates the 
Application object. 


Run the following command to inspect that the change has been detected and 
registered: 
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kubectl describe applicationset myapps -n argocd 


Events: 


Type Reason Age From Message 
Normal created 23s applicationset-controller created Applica- 


tion "myapp-lordofthejars-patch-1-1" 
Normal unchanged 23s (x2 over 23s) applicationset-controller unchanged Appli- 
cation "myapp-lordofthejars-patch-1-1" 


Check the registration of the Application to the pull request: 


argocd app list 


NAME CLUSTER NAMESPACE 
myapp- Lordofthejars-patch-1-1 https://kubernetes.default.svc lordofthejars- 
patch-1-1 


The Application object is automatically removed when the pull request is closed. 


Discussion 
At the time of writing this book, the following pull request providers are supported: 
e GitHub 
e Bitbucket 
e Gitea 
e GitLab 


The ApplicationSet controller polls every requeueAfterSeconds interval to detect 
changes but also supports using webhook events. 


To configure it, follow Recipe 8.3, but also enable sending pull requests events too in 
the Git provider. 


8.6 Use Advanced Deployment Techniques 


Problem 


You want to deploy the application using an advanced deployment technique such as 
blue-green or canary. 


Solution 


Use the Argo Rollouts project to roll out updates to an application. 
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Argo Rollouts is a Kubernetes controller providing advanced deployment techniques 
such as blue-green, canary, mirroring, dark canaries, traffic analysis, etc. to Kuber- 
netes. It integrates with many Kubernetes projects like Ambassador, Istio, AWS Load 
Balancer Controller, NGNI, SMI, or Traefik for traffic management, and projects 
like Prometheus, Datadog, and New Relic to perform analysis to drive progressive 
delivery. 


To install Argo Rollouts to the cluster, run the following command in a terminal 
window: 


kubectl create namespace argo-rollouts 


kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/relea- 
ses/download/v1.2.2/install. yaml 


clusterrolebinding.rbac.authorization.k8s.io/argo-rollouts created 
secret/argo-rollouts-notification-secret created 
service/argo-rollouts-metrics created 
deployment.apps/argo-rollouts created 


Although it’s not mandatory, we recommend you install the Argo Rollouts Kubectl 
Plugin to visualize rollouts. Follow the instructions to install it. With everything in 
place, let’s deploy the initial version of the BGD application. 


Argo Rollouts doesnt use the standard Kubernetes Deployment file, but a specific 
new Kubernetes resource named Rollout. It’s like a Deployment object, hence all its 
options are supported, but it adds some fields to configure the rolling update. 


Lets deploy the first version of the application. We'll define the canary release process 
when Kubernetes executes a rolling update, which in this case follows these steps: 


1. Forward 20% of traffic to the new version. 
2. Wait until a human decides to proceed with the process. 


3. Forward 40%, 60%, 80% of the traffic to the new version automatically, waiting 
30 seconds between every increase. 


Create a new file named bgd-rollout.yaml with the following content: 


apiVersion: argoproj.io/vialpha1 
kind: Rollout 
metadata: 
name: bgd-rollouts 
spec: 
replicas: 5 
strategy: 
canary: © 
steps: © 
- setWeight: 20 © 
- pause: {} (4) 
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- setWeight: 40 
- pause: {duration: 30s} (5) 
- setWeight: 60 
- pause: {duration: 30s} 
- setWeight: 80 
- pause: {duration: 30s} 
revisionHistoryLimit: 2 
selector: 
matchLabels: 
app: bgd-rollouts 
template: (6) 
metadata: 
creationTimestamp: null 
labels: 
app: bgd-rollouts 
spec: 
containers: 
- image: quay.io/rhdevelopers/bgd:1.0.0 
name: bgd 
env: 
- name: COLOR 
value: "blue" 
resources: {} 


Canary release 

List of steps to execute 
Sets the ratio of canary 
Rollout is paused 


Pauses the rollout for 30 seconds 


O © 6 © 8 8 


template Deployment definition 


Apply the resource to deploy the application. Since there is no previous deployment, 
the canary part is ignored: 


kubectl apply -f bgd-rollout.yaml 
Currently, there are five pods as specified in the replicas field: 


kubectl get pods 


NAME READY STATUS RESTARTS AGE 
bgd-rollouts-679cdfcfd-6z2zf 1/1 Running 0 12m 
bgd-rollouts-679cdfcfd-8c6kl 1/1 Running 0 12m 
bgd-rollouts-679cdfcfd-8tb4v 1/1 Running 0 12m 
bgd-rollouts-679cdfcfd-f4p7f 1/1 Running 0 12m 
bgd-rollouts-679cdfcfd-tljfr 1/1 Running 0 12m 
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And using the Argo Rollout Kubectl Plugin: 


kubectl argo rollouts get rollout bgd-rollouts 


Name: bgd-rollouts 
Namespace: default 
Status: V Healthy 
Strategy: Canary 
Step: 8/8 
SetWeight: 100 
ActualWeight: 100 
Images: quay.io/rhdevelopers/bgd:1.0.0 (stable) 
Replicas: 
Desired: 5 
Current: 5 
Updated: 5 
Ready: 5 
Available: 5 
NAME KIND STATUS 
C bgd-rollouts Rollout v Healthy 
L—# revision:1 
Lt bgd-rollouts-679cdfcfd ReplicaSet JW Healthy 
2 bgd-rollouts-679cdfcfd-6z2zf Pod v¥ Running 
o bgd-rollouts-679cdfcfd-8c6kl Pod v¥ Running 
o bgd-rollouts-679cdfcfd-8tb4v Pod V Running 
}—0 bgd-rollouts-679cdfcfd-f4p7f Pod v¥ Running 
L—o bgd-rollouts-679cdfcfd-tljfr Pod v¥ Running 


AGE 
13m 


13m 
13m 
13m 
13m 
13m 
13m 


INFO 


stable 

ready:1/1 
ready:1/1 
ready:1/1 
ready:1/1 
ready:1/1 


Let’s deploy a new version to trigger a canary rolling update. Create a new file named 
bgd-rollout-v2.yaml with exactly the same content as the previous one, but change the 
environment variable COLOR value to green: 


name: bgd 


env: 
- name: COLOR 


value: "green" 
resources: {} 


Apply the previous resource and check how Argo Rollouts executes the rolling 
update. List the pods again to check that 20% of the pods are new while the other 80% 


are the old version: 


kubectl get pods 


NAME 

bgd-rollouts-679cdfcfd-6z2zf 
bgd-rollouts-679cdfcfd-8c6kl 
bgd-rollouts-679cdfcfd-8tb4v 
bgd-rollouts-679cdfcfd-tljfr 
bgd-rollouts-c5495c6ff-zfgvn 


READY 
1/1 
t/i 
1/1 
1/1 
1/1 


STATUS 

Running 
Running 
Running 
Running 
Running 


RESTARTS AGE 


oooo © 


27m 
27m 
27m 
27m 


13s © 
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@ New version pod 
And do the same using the Argo Rollout Kubectl Plugin: 


kubectl argo rollouts get rollout bgd-rollouts 


NAME KIND STATUS AGE INFO 


c bgd-rollouts Rollout ll Paused 31m 

ļ—# revision:2 

| I bgd-rollouts-c5495c6ff ReplicaSet W Healthy 3m21s canary 

| L—o bgd-rollouts-c5495c6ff-zfgvn Pod ¥ Running 3m21s ready:1/1 

L—# revision:1 

Lt bgd-rollouts-679cdfcfd ReplicaSet VW Healthy 31m stable 

o bgd-rollouts-679cdfcfd-6z2zf Pod ¥ Running 31m ready:1/1 
}—0 bgd-rollouts-679cdfcfd-8c6kl Pod V Running 31m ready:1/1 
o bgd-rollouts-679cdfcfd-8tb4v Pod ¥ Running 31m ready:1/1 


L—o bgd-rollouts-679cdfcfd-tljfr Pod ¥ Running 31m ready:1/1 


Remember that the rolling update process is paused until the operator executes a 
manual step to let the process continue. In a terminal window, run the following 
command: 


kubectl argo rollouts promote bgd-rollouts 


The rollout is promoted and continues with the following steps, which is substituting 
the old version pods with new versions every 30 seconds: 


kubectl get pods 


NAME READY STATUS RESTARTS AGE 
bgd-rollouts-c5495c6ff-2g7r8 1/1 Running 0 89s 
bgd-rollouts-c5495c6ff-7mdch 1/1 Running 0 122s 
bgd-rollouts-c5495c6ff-d9828 1/1 Running 0 13s 
bgd-rollouts-c5495c6ff-h4t6f 1/1 Running 0 56s 
bgd-rollouts-c5495c6ff-zfgvn 1/1 Running 0 11m 


The rolling update finishes with the new version progressively deployed to the cluster. 


Discussion 


Kubernetes doesnt implement advanced deployment techniques natively. For this 
reason, Argo Rollouts uses the number of deployed pods to implement the canary 
release. 


As mentioned before, Argo Rollouts integrates with Kubernetes products that offer 
advanced traffic management capabilities like Istio. 


Using Istio, the traffic splitting is done correctly at the infrastructure level instead of 
playing with replica numbers like in the first example. Argo Rollouts integrates with 
Istio to execute a canary release, automatically updating the Istio VirtualService 
object. 


212 | Chapter 8: Advanced Topics 


Assuming you already know Istio and have a Kubernetes cluster with Istio installed, 
you can perform integration between Argo Rollouts and Istio by setting the 
trafficRouting from Rollout resource to Istio. 


First, create a Rollout file with Istio configured: 


apiVersion: argoproj.io/vialpha1 
kind: Rollout 
metadata: 
name: bgdapp 
labels: 
app: bgdapp 
spec: 
strategy: 
canary: 
steps: 
- setWeight: 20 
- pause: 
duration: "1m" 
- setWeight: 50 
- pause: 
duration: "2m" 
canaryService: bgd-canary (2) 
stableService: bgd © 
trafficRouting: 
istio: O 
virtualService: @ 
name: bgd Q 
routes: 
- primary @ 
replicas: 1 
revisionHistoryLimit: 2 
selector: 
matchLabels: 
app: bgdapp 
version: v1 
template: 
metadata: 
labels: 
app: bgdapp 
version: vi 
annotations: 
sidecar.istio.io/inject: "true" 8] 
spec: 
containers: 
- image: quay.io/rhdevelopers/bgd:1.0.0 
name: bgd 
env: 
- name: COLOR 
value: "blue" 
resources: {} 
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Canary section 

Reference to a Kubernetes Service pointing to the new service version 
Reference to a Kubernetes Service pointing to the old service version 
Configures Istio 

Reference to the VirtualService where weight is updated 

Name of the VirtualService 


Route name within VirtualService 


O © O O © 86 8 8 


Deploys the Istio sidecar container 


Then, we create two Kubernetes Services pointing to the same deployment used to 
redirect traffic to the old or the new one. 


The following Kubernetes Service is used in the stableService field: 


apiVersion: v1 
kind: Service 
metadata: 
name: bgd 
labels: 
app: bgdapp 
spec: 
ports: 
- name: http 
port: 8080 
selector: 
app: bgdapp 


And the Canary one is the same but with a different name. Its the one used in the 
canaryService field: 


apiVersion: vi 
kind: Service 
metadata: 
name: bgd-canary 
labels: 
app: bgdapp 
spec: 
ports: 
- name: http 
port: 8080 
selector: 
app: bgdapp 
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Finally, create the Istio Virtual Service to be updated by Argo Rollouts to update the 
canary traffic for each service: 


(2) 
© 


apiVersion: networking.istio.io/vialpha3 
kind: VirtualService 
metadata: 
name: bgd 
spec: 
hosts: 
- bgd 
http: 
- route: 
- destination: 
host: bgd @ 
weight: 100 
- destination: 
host: bgd-canary (2) 
weight: 0 
name: primary 


Stable Kubernetes Service 
Canary Kubernetes Service 


Route name 


After applying these resources, we'll get the first version of the application up and 
running: 


When any 


kubectl apply -f bgd-virtual-service. yaml 


kubectl apply -f service.yaml 
kubectl apply -f service-canary.yaml 
kubectl apply -f bgd-isio-rollout.yaml 


update occurs on the Rollout object, the canary release will start as 


described in the Solution. Now, Argo Rollouts updates the bgd virtual service weights 
automatically instead of playing with pod numbers. 


See Also 


Argo Rollouts - Kubernetes Progressive Delivery Controller 


Istio - Argo Rollouts 
Istio 


Istio Tutorial from Red Hat 
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